Moving picture encoding method, moving picture decoding method, and recording medium

ABSTRACT

In coding a picture to be used as a reference picture by another picture, a motion vector used in inter picture predictive coding is stored into the motion vector storage unit ( 116 ). However, the storage of a motion vector is controlled by the coding control unit ( 110 ) in such a way that motion vectors of pictures, fewer in number than the number of reference pictures, are stored into the storage unit ( 116 ). In the case where direct mode is used in coding a B-picture, if the motion vector to be referred to in coding in direct mode is stored in the motion vector storage unit ( 116 ), coding is performed using such motion vector. Further, in the case where the motion vector is not stored in the storage unit ( 116 ), coding is performed with the motion vector as “0”, and using a motion vector of a neighboring block.

TECHNICAL FIELD

The present invention relates to a picture coding method for theefficient compression of moving pictures and a picture decoding methodfor correctly decoding the compressed moving pictures, and particularlyto a picture coding method and picture decoding method using direct modefor processing.

BACKGROUND ART

In the age of multimedia which integrally handles audio, video and otherinformation, existing information media, i.e., newspaper, magazine,television, radio, telephone and other means through which informationis conveyed to people, have recently come to be included in the scope ofmultimedia. Generally, multimedia refers to something that isrepresented by associating not only characters, but also graphics,sound, and especially images, and the like, together, but in order toinclude the aforementioned existing information media in the scope ofmultimedia, it becomes a prerequisite to represent such information indigital form.

However, if we appraise the amount of information carried by each of theaforementioned information media as the amount of digital information,while the amount of information for 1 character in the case of text is 1to 2 bytes, the amount of information required for voice is 64Kbits persecond (telephone quality), and 100Mbits or over per second becomesnecessary for moving pictures (current television reception quality), itis not realistic for the aforementioned information media to handle suchan enormous amount of information as it is in digital form. For example,although video phones are already in actual use via Integrated ServicesDigital Network (ISDN) which offers a transmission speed of 64Kbps/s to1.5Mbps/s, it is not practical to transmit video shot by televisioncameras directly through ISDN.

Against this backdrop, information compression techniques have becomerequired, and for example, in the case of the video phone, the H.261 andH.263 standards for moving picture compression technology,internationally standardized by the International TelecommunicationUnion—Telecommunication Standardization Sector (ITU-T), are beingemployed. Moreover, with MPEG-1 standard information compressiontechniques, it has also become possible to store video information ontonormal music compact discs (CD) together with audio information.

Here, Moving Picture Experts Group (MPEG) is an international standardfor moving picture signal compression, and MPEG-1 is a standard forcompressing moving picture signals up to 1.5Mbps, in other words,compressing television signals up to approximately a hundredth of theoriginal size. Moreover, since transmission speed within the scope ofthe MPEG-1 standard is limited primarily to about 1.5Mbps, the use ofMPEG-2, which was standardized to satisfy demands for further improvedpicture quality, allows moving picture signals to be compressed to 2 to15Mbps. Furthermore, at present, MPEG-4, which has exceeded MPEG-1 andMPEG-2 compression ratios, and also enables coding, decoding andoperating on a per-object base, and realizes the new functions requiredfor the multimedia age, has been standardized by the work group (ISO/IECJTC1/SC29/WG11) that has promoted the standardization of MPEG-1 andMPEG-2. MPEG-4 was initially aimed at standardizing a low bit ratecoding method, but at present, this has been expanded to thestandardization of a more versatile coding method further including highbit rate coding for interlaced images, and others.

As for B-picture (hereinafter referred to as “picture”, whenever a stillpicture, which is one picture within a moving picture, can either be a“frame” or “field”) coding in the case of moving picture coding methodsfor MPEG-4, H.26L, and others, a coding mode known as Direct Mode can beselected. (See MPEG-4 visual written standards (1999, ISO/IEC14496-2:1999 Information technology—Coding of audio-visual objects—Part2: Visual, p.154)). FIG. 1 is a diagram showing an example of an interpicture prediction method in an existing direct mode. FIG. 1 shall bereferred to in explaining an inter picture prediction coding method indirect mode. It is now assumed that a block “a” of a picture B3 iscoded/decoded in direct mode. In this case, when the picture B3 iscoded/decoded in the H.26L standard, the motion vectors of a block,co-located with the block “a”, within the reference picture whose secondreference index (a reference index is also referred to as a “relativeindex”. A reference index will be discussed later) is “0”, shall beused. Here, it is assumed that with regard to the picture B3, a pictureP4 is the reference picture whose second reference index is “0”. In thiscase, a motion vector “c” of the block “b” within the picture P 4 shallbe used. The motion vector “c” is the motion vector used during thecoding/decoding of the block “b”, and refers to a picture P1. For theblock “a”, a bi-prediction from the reference pictures P1 and P4 iscarried out using a motion vector parallel to the motion vector “c”. Themotion vectors in this case where the block “a” is coded/decoded shallbe a motion vector “d” for the picture P1, and a motion vector “e” forthe picture 4.

FIG. 2 is a chart showing an example of the assignment of picturenumbers, as well as reference indices for each picture inputted. Picturenumber and reference indices are numbers for uniquely identifyingreference pictures stored in the reference picture memory. For eachpicture stored in memory as a reference picture, a number incrementingby the value of “1” is assigned as a picture number.

FIG. 3 is a conceptual diagram showing the format of a picture codedsignal in an existing moving picture coding, moving picture decodingmethod. “Picture” stands for a coded signal for one picture, “Header” isa header coded signal included in a picture head, “Block 1” is the codedsignal of a block coded by direct mode, “Block 2” is the coded signal ofa block by an interpolation (motion compensation) prediction other thandirect mode, “Rldx0” and “Rldx1” are reference indices, “MV0” and “MV 1”represent motion vectors. For the interpolation (motion compensation)predictive block, Block 2, the two reference indices for indicating thetwo reference pictures (a first reference picture and a second referencepicture) used for interpolation (motion compensation), Rldx0 and Rldx1,are contained, in this order, within a coded signal. Which of thereference indices Rldx0 or Rldx1 shall be used can be determined basedon PredType. For example, in the case where PredType has indicated thata picture shall be referred bi-directionally, Rldx0 and Rldx1 areapplied. Where it is indicated that a picture shall be uni-directionallyreferred, either Rldx0 or Rldx1 is applied, and in the case where directmode is indicated, neither Rldx0 nor Rldx1 is applied. The referenceindex Rldx1, which indicates the first reference picture, shall be knownas the first reference index and, the reference index Rldx1, indicatingthe second reference picture, shall be known as the second referenceindex. The first and second reference pictures are identified based onthe position of data in a bit stream.

From here, FIG. 2A is referred to in explaining a method for assigningthe first and second reference indices.

For the value of the first reference index, first, values starting from“0” shall be assigned to reference pictures having a display timeearlier than a current picture to be coded/decoded, in the order ofproximity to the current picture to be coded/decoded. When all thereference pictures having a display time earlier than the currentpicture to be coded/decoded have been assigned values starting from “0”,the continuing values are then assigned to the reference pictures havinga display time later than the current picture to be coded/decoded, inthe order of proximity to the current picture to be coded/decoded.

For the value of the second reference index, first, values starting from“0” shall be assigned to reference pictures having a display time laterthan the current picture to be coded/decoded, in the order of proximityto the current picture to be coded/decoded. When all the referencepictures having a display time later than the current picture to becoded/decoded have been assigned values starting from “0”, thecontinuing values are then assigned to the reference pictures having adisplay time earlier than the current picture to be coded/decoded, inthe order of proximity to the current picture to be coded/decoded.

In FIG. 2A, where a first reference index Rldx0 is “0” and a secondreference index Rldx1 is “1”, the first reference picture is a B-picturewith a picture number “14”, and the second reference picture is aB-picture with a picture number “13”.

A reference index within a block is expressed by variable length codewords, where the smaller the value of the reference index is, theshorter the code length of the code assigned is. Since, normally, thepossibility of the picture closest to the current picture to becoded/decoded being chosen as a reference picture for inter pictureprediction is high, coding efficiency will increase if the referenceindex values are assigned in the order of proximity to the currentpicture to be coded/decoded, as described above.

On the other hand, by indicating a change (remapping) in the assignmentof reference indices using the buffer control signal within the codedsignal (See FIG. 3, RPSL within Header), it is possible to arbitrarilychange the reference picture assignment for the reference indices.Accordingly, with this change of assignment, it now becomes acceptableto appoint any reference picture within the picture memory, as thereference picture with a second reference index as “0”. For example, asshown in FIG. 2B, it is also possible to change the reference indexassignment for picture numbers, so as to allow a reference picture witha second reference index as “0” to become the reference picture having adisplay time immediately preceding the current picture to becoded/decoded.

In addition, in the example given in FIG. 2A and FIG. 2B, a case where aB-picture is referred to during the coding/decoding of another pictureis shown, but in general, coding is more often performed under theconditions listed below.

-   -   (1) A B-picture is not to be referred to by another picture.    -   (2) For each block of a B-picture, motion compensation is        performed with reference to two pictures which are arbitrarily        chosen as reference pictures from among the N (N being a        positive integer) number of P-pictures (or I-pictures)        immediately preceding in display order and, a single P-picture        (or I-picture) immediately subsequent in display order.

FIG. 4A is a diagram showing an example default setting of referenceindices for a current picture to be coded B11, where a B-picture iscoded with four preceding (N=4) P-pictures and a single, subsequentP-picture as reference pictures. In FIG. 4A, the difference with theexample shown in FIG. 2A is that, as B-pictures are not referred to byother pictures, no reference indices are assigned to B-pictures and,only P-pictures (and I-pictures) are assigned reference indices. Forexample, since the picture B11 can use the four P-pictures immediatelypreceding it in display order, and the single P-picture immediatelysubsequent as reference pictures, reference indices will only beassigned to a picture P0, a picture P1, a picture P4, a picture P7, aswell as a picture P10.

In the example shown in FIG. 4A, for the picture B11, the referencepicture with a first reference index as “0” is the picture P7, and thereference picture with a second reference index as “0” is the pictureP10. The picture P10 is located after the picture B11 in display order,and is the closest P-picture to the picture B11. Even under theabove-mentioned conditions, it is possible to flexibly change theassignment of reference pictures for reference indices. FIG. 4B is adiagram showing an example of the reference indices for the picture B11in the case where a remapping has been performed on the referenceindices shown in FIG. 4A. As shown in FIG. 4B, in the H.26L standard, itis possible to re-assign the value “0” of the first reference indexassigned to the picture P7 in the default setting, to the picture P1,and likewise, to re-assign the value “0” for the second reference indexassigned to the picture P10 in the default setting, to the picture P0.It is possible to remap reference indices freely based on the codingefficiency, and other factors, of the subject B-picture.

In this manner, since it is possible to freely change the assignment ofreference indices for reference pictures, a change can be normally doneso that, the picture that is chosen to be the reference picture forimproving the coding efficiency of a current picture to be coded, can beassigned an even smaller reference index. In other words, since areference index within a block is expressed by variable length codewords, where the code length of the code to be assigned becomes shorteras the value becomes smaller, the assignment of an even smallerreference index to a picture to which reference allows improvement ofcoding efficiency reduces the amount of coding for a reference index,thus allowing a further improvement in coding efficiency.

In the above-mentioned existing method, the motion vector of thereference picture with a second reference index as “0” is used forprocessing a block in a B-picture in direct mode. As such, during theprocess of coding/decoding a B-picture, it becomes necessary to storethe motion vector of the reference picture with a second reference indexas “0”. However, during the decoding process, in particular, until theprocessing of the bit stream of a current B-picture to be decoded isstarted, it is not known which reference picture is the picture with asecond reference index as “0”. This is because it is possible toarbitrarily change the assignment of reference indices for the referencepictures, through the explicit instruction of the buffer control signal(See FIG. 3, RPSL within Header). Accordingly, in the process ofcoding/decoding a B-picture, it becomes necessary to store the motionvectors of all reference pictures. As such, in an existing method, aspicture size gets larger, and, as reference pictures increase in number,there is a problem of an explosive expansion of memory size needed forstoring motion vectors.

The present invention is conceived to solve the above-mentioned problem,and the object thereof is to provide a moving picture coding method anda moving picture decoding method for direct mode, that enables reductionof memory size for motion vectors.

DISCLOSURE OF INVENTION

In order to solve this problem, the moving picture coding methodaccording to the present invention is a moving picture coding method forcoding a moving picture, the moving picture being a sequence ofpictures, each of which is made up of a plurality of blocks, the methodincluding: a step of coding a moving picture on a per-block basis withmotion compensation using a motion vector, and generating a bit stream;a step of generating limitation information indicating a limitation onstorage of at least one motion vector used in motion compensation; and astep of coding the generated limitation information. As a result, it ispossible to show a limitation on the storage of motion vector used inperforming motion compensation on a current block to be coded within apicture containing a plurality of blocks.

Furthermore, another moving picture coding method in the presentinvention is a moving picture coding method for coding a moving picturewith motion compensation using a motion vector, the moving picture beinga sequence of pictures, each of which is made up of a plurality ofblocks, the method including: a determination step of determining acoding mode for a current block to be coded; a judgment step of judgingwhether or not a motion vector of a coded picture identified by areference index is stored in a storage unit, in the case where thecoding mode for the current block to be coded is direct mode; and acompensation step of performing motion compensation on a current blockto be coded using a motion vector used in coding a neighboring block ofthe current block to be coded within the same picture containing thecurrent block to be coded, in the case where the motion vector of thecoded picture identified by the reference index is judged as not beingstored in the storage unit. Here, direct mode is a coding mode forperforming motion compensation on the current block to be coded using amotion vector used in coding of a block within a coded picture. Thereference index is an index assigned to the coded picture for selectinga reference picture to be referred to in performing motion compensationon the current block to be coded, from among a plurality of codedpictures stored in the storage unit. As a result, it is possible toperform motion compensation on the current block to be coded even if themotion vector necessary for performing motion compensation in directmode is not in the storage unit.

Moreover, the above-mentioned moving picture coding method may furtherinclude another compensation step of performing motion compensation onthe current block to be coded with the motion vector of the currentblock to be coded as “0”, in the case where the motion vector of thecoded picture identified by the reference index is judged as not beingstored in the storage unit. As a result, it is possible to performmotion compensation on the current block to be coded even if the motionvector necessary for performing motion compensation in direct mode isnot in the storage unit.

Furthermore, the above-mentioned moving picture coding method mayfurther include still another compensation step of performing motioncompensation on the current block to be coded using the motion vectorstored in the storage unit, in the case where the motion vector of thecoded picture identified by the reference index is judged as beingstored in the storage unit. As a result, it is possible to performmotion compensation on a current block to be coded based on the judgmentof whether the motion vector necessary for performing motioncompensation in direct mode is stored or not.

Also, still another moving picture coding method in the presentinvention is a moving picture coding method for coding a moving picture,the moving picture being a sequence of pictures, each of which is madeup of a plurality of blocks, the method including: a step of coding amoving picture on a per-block basis with motion compensation using amotion vector, and generating a bit stream; a first output step ofgenerating limitation information indicating a limitation on storage ofat least one motion vector used in motion compensation; a second outputstep of generating storage information, in the case where a picturereferred to by the current block to be coded is a B-picture, saidstorage information indicating that a motion vector used in coding ofsaid B-picture, is not to be stored into a storage unit; and a step ofcoding the generated limitation information and the generated storageinformation. Here, the B-picture is a picture on which motioncompensation is performed on a per-block basis referring to, at most,two pictures. As a result, it is possible to indicate a limitation onthe storage of motion vectors used in performing motion compensation ona current block to be coded within a picture containing a plurality ofblocks, and indicate, according to type, the motion vectors that are notto be stored.

In the step of coding the limitation information and the storageinformation in the above-mentioned moving picture coding method, saidlimitation information and said storage information may be coded asheader information within the bit stream. The above-mentioned movingpicture coding method further includes a step of storing, into a storagearea, a motion vector of a P-picture coded immediately before a picturecontaining the current block to be coded, in coding order, withoutstoring, into the storage area, the motion vector used in coding theB-picture, based on the storage information. Here, the P-picture may bea picture on which predictive coding is performed on a per-block basisreferring to, at most, one coded picture. As a result, it is possible toindicate a limitation on the storage of motion vectors used inperforming motion compensation on a current block to be coded within apicture containing a plurality of blocks, and indicate, according totype, the motion vectors that are not to be stored.

Still another moving picture coding method in the present invention is amoving picture coding method for coding a moving picture with motioncompensation using a motion vector, the moving picture being a sequenceof pictures, each of which is made up of a plurality of blocks, themethod including: a step of judging whether a current picture to becoded is a top field or a bottom field, in the case where said currentpicture to be coded is one of the fields of an interlaced picture,composed of said top field and said bottom field; and a step of storing,into a top field storage area, the motion vector used in motioncompensation of the current picture to be coded, in the case where saidcurrent picture to be coded is judged to be the top field, and storing,into a bottom field storage area, the motion vector used in motioncompensation of the current picture to be coded, in the case where saidcurrent picture to be coded is judged to be the bottom field. As aresult, it is possible to store motion vectors of fields, on aper-parity basis.

In the above-mentioned coding method, the motion vector to be storedinto the top field storage area and the motion vector to be stored intothe bottom field storage area may be the motion vectors used in codingof the top field and the bottom field belonging to the same frame. As aresult, it is possible to store the motion vectors used in coding fieldsbelonging to the same frame.

Furthermore, the above-mentioned coding method may further include astep of storing the motion vectors used in motion compensation of thecurrent picture to be coded, into the top field storage area and thebottom field storage area, in the case where the current picture to becoded is coded in a frame structure. As a result, it is possible tostore, into both the top field storage area and the bottom field storagearea, the motion vectors used in the motion compensation of a currentpicture in the frame structure.

In order to solve the above-mentioned problem, the moving picturedecoding method according to the present invention is a moving picturedecoding method for decoding a moving picture with motion compensationusing a motion vector, the moving picture being a sequence of pictures,each of which is made up of a plurality of blocks, the method including:a step of inputting a bit stream containing limitation informationindicating a limitation on storage of at least one motion vector used inmotion compensation of a current block to be decoded; a step ofextracting the limitation information after decoding the bit stream; anda step of storing said at least one motion vector into a storage areabased on the limitation information. As a result, it is possible tolimit the storage of motion vector used in performing motioncompensation on a current block to be coded within a picture containinga plurality of blocks.

Also, another moving picture decoding method in the present invention isa moving picture decoding method for decoding a moving picture withmotion compensation using a motion vector, the moving picture being asequence of pictures, each of which is made up of a plurality of blocks,the method including: a step of inputting a bit stream containing codingmode information indicating a coding mode of a current block to bedecoded; a step of extracting the coding mode information after decodingthe bit stream; a judgment step of judging whether or not a motionvector used in decoding a decoded picture identified by a referenceindex, is stored in a storage unit, in the case where the coding modeinformation indicates direct mode; and a compensation step of performingmotion compensation on the current block to be decoded using a motionvector used in decoding a neighboring block of the current block to bedecoded within the same picture containing the current block to bedecoded, in the case where the motion vector of the decoded pictureidentified by the reference index is judged as not being stored in thestorage unit. Here, the direct mode is a coding mode for performingmotion compensation on the current block to be decoded using a motionvector used in decoding a block, co-located with the current block to bedecoded, within a picture being referred to. The reference index is anindex assigned to the decoded picture for selecting the referencepicture to be referred to for performing motion compensation on thecurrent block to be decoded, from among a plurality of decoded picturesstored in the storage unit. As a result, it is possible to performmotion compensation on the current block to be decoded even when themotion vector necessary for performing motion compensation in directmode is not in the storage unit.

Also, in addition, the above-mentioned moving picture decoding methodmay further include another compensation step of performing motioncompensation on the current block to be decoded with the motion vectorof the current block to be decoded as “0”, in the case where the motionvector of the decoded picture identified by the reference index isjudged as not being stored in the storage unit. As a result, it ispossible to perform motion compensation on the current block to bedecoded even when the motion vector necessary for performing motioncompensation in direct mode is not in the storage unit.

Furthermore, the above-mentioned decoding method may further includestill another compensation step of performing motion compensation on thecurrent block to be decoded using the motion vector stored in thestorage unit, in the case where the motion vector used in decoding thedecoded picture identified by the reference index, is judged as beingstored in the storage unit. As a result, it is possible to performmotion compensation on a current block to be decoded based on thejudgment of whether the motion vector necessary for performing motioncompensation in direct mode is stored or not.

Still another decoding method in the present invention is a movingpicture decoding method for decoding a moving picture with motioncompensation using a motion vector, the moving picture being a sequenceof pictures, each of which is made up of a plurality of blocks, themethod including: an input step of inputting a bit stream containinglimitation information and storage information, said limitationinformation indicating a limitation on storage of at least one motionvector used in motion compensation of a current block to be decoded, andsaid storage information indicating that a motion vector used indecoding a B-picture is not stored in the storage unit when a picturereferred to by the current block to be decoded is the B-picture; anextraction step of extracting said limitation data and said storage dataafter decoding the bit stream; and a storage step of storing, into thestorage unit, the motion vector, based on the limitation information,without storing, into the storage unit, the motion vector used indecoding the B-picture, based on the storage information. Here, theB-picture is a picture on which motion compensation is performed on aper-block basis referring to, at most, 2 decoded pictures. As a result,it is possible to determine whether a motion vector is to be stored ornot, based on the extracted limitation information and storageinformation.

Furthermore, in the above-mentioned storage step, a motion vector of aP-picture decoded immediately before a picture containing the currentblock to be decoded, in decoding order, is stored into the storage area,without storing the motion vector used in decoding the B-picture intothe storage area, based on the storage information. The P-picture may bea picture on which motion compensation is performed on a per-block basisreferring to, at most, one decoded picture. As a result, it is possibleto determine whether a motion vector is to be stored or not, based onthe extracted limitation information and storage information.

Also, still another decoding method in the present invention is a movingpicture decoding method for decoding a moving picture with motioncompensation using a motion vector, the moving picture being a sequenceof pictures, each of which is made up of a plurality of blocks, themethod including: a step of judging whether a current picture to bedecoded is a top field or a bottom field, in the case where said currentpicture to be decoded is one of the fields of an interlaced picture,composed of said top field and said bottom field; and a step of storing,into a top field storage area, the motion vector used in motioncompensation of the current picture to be decoded, in the case wheresaid current picture to be decoded is judged to be the top field, andstoring, into a bottom field storage area, the motion vector used inmotion compensation of the current picture to be decoded, in the casewhere said current picture to be decoded is judged to be the bottomfield. As a result, it is possible to store motion vectors of fields, ona per-parity basis.

As mentioned above, according to the moving picture coding method andthe moving picture decoding method in the present invention, it ispossible to generate a bit stream with a higher coding efficiency, whilereducing the memory size for storing motion vectors necessary for codingin direct mode. Thus the practical value of the present invention ishigh.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing an example of an inter picture predictionmethod in an existing direct mode.

FIG. 2 is an explanatory chart showing an example of the assignment ofpicture numbers, as well as reference indices for each picture inputted.

FIG. 3 is a conceptual diagram showing the format of a picture codedsignal in an existing moving picture coding, moving picture decodingmethod.

FIG. 4A is a diagram showing an example default setting of referenceindices for a current picture to be coded B11, where a B-picture iscoded with four preceding P-pictures and a single subsequent P-pictureas reference pictures.

FIG. 4B is a diagram showing an example of the reference indices for thepicture B11 in the case where a remapping has been performed on thereference indices shown in FIG. 4A.

FIG. 5 is a block diagram showing the configuration of a moving picturecoding apparatus in the first embodiment of the present invention.

FIG. 6A is a diagram showing the sequence of pictures inputted as inputpictures into the moving picture coding apparatus in FIG. 5.

FIG. 6B is a diagram showing the sequence of pictures when the sequenceof pictures indicated in FIG. 6A is reordered into coding order.

FIG. 7A to 7D are diagrams showing an example of a bit stream generatedby the bit stream generation unit shown in FIG. 5.

FIG. 7A is a diagram showing an example of a bit stream in whichinformation regarding the storage size of motion vectors is described ina sequence header.

FIG. 7B is a diagram showing an example of a bit stream in whichinformation regarding the storage size of motion vectors is described ina Group of Pictures (GOP) header.

FIG. 7C is a diagram showing an example of a bit stream in whichinformation regarding the storage size of motion vectors is described ina picture header.

FIG. 7D is a diagram showing an example a bit stream in whichinformation regarding the storage size of motion vectors is described ina slice header.

FIG. 8 is a diagram showing an example of a calculation method in thecase where a motion vector of a current block is calculated using directmode.

FIG. 9 is a diagram showing an example of a bi-prediction in the casewhere, during the coding of a B-picture in direct mode, the motionvector of the reference picture with a second reference index as “0”cannot be used.

FIG. 10 is a diagram showing another example of a bi-prediction in thecase where, during the coding of a B-picture in direct mode, the motionvector of the reference picture with a second reference index as “0”cannot be used.

FIG. 11 is a block diagram showing the configuration of the movingpicture decoding apparatus in the second embodiment.

FIG. 12A is a diagram showing the sequence of pictures included within abit stream.

FIG. 12B is a diagram showing the sequence of the pictures shown in FIG.12A, when reordered into decoding order.

FIG. 13 is a block diagram showing the outline configuration of arecording apparatus, for recording onto a recording medium such as DVD,the management information for managing the reproduction of data fromthe recording medium and coded Audio Visual (AV) data.

FIG. 14 is an illustration showing an example of a recording medium 400such as DVD, on which the management information and Audio Video (AV)streams generated by the system coding unit shown in FIG. 13 arerecorded.

FIG. 15 is a block diagram showing the configuration of a moving picturecoding apparatus included within the recording apparatus shown in FIG.13.

FIG. 16A is a diagram showing pictures inputted in display order, andtheir respective coding order.

FIG. 16B is a diagram showing the reference pictures and motion vectorsstored into the memory following the coding of each picture in theexisting MPEG-4 format.

FIG. 17A is a diagram showing pictures inputted in display order, andtheir respective coding order.

FIG. 17B is a diagram showing the reference pictures and motion vectorsstored into the memory following the coding of each picture in themoving picture coding apparatus shown in FIG. 15.

FIG. 18 is a diagram showing an example of a data stream sent out duringdigital television broadcasts.

FIG. 19 is a block diagram showing the outline configuration of areproduction apparatus in the fourth embodiment.

FIG. 20 is a block diagram showing the configuration of a moving picturedecoding apparatus that corresponds to the video decoding unit shown inFIG. 19.

FIG. 21 is a diagram showing an example of the reference relationshipbetween pictures in the case where the current picture to be coded ordecoded is in the field structure.

FIG. 22 is a diagram showing an example of the reference relationshipbetween pictures in the case where the current picture to be coded ordecoded is a frame.

FIG. 23 is a diagram showing the temporal sequence of respective framesduring the coding or decoding of a moving picture.

FIG. 24A is a schematic diagram for explaining the operation of amemory, in the case where the memory address for storing motion vectorsis for storing motion vectors of one picture at a time.

FIG. 24B is a schematic diagram for explaining the operation of amemory, in the case where the motion vectors of a top field and motionvectors of a bottom field are stored, two fields at a time.

FIG. 25A shows an example of a physical format of the flexible disk, asa recording medium itself.

FIG. 25B shows the front view of a flexible disk, a schematiccross-section, as well as a flexible disk.

FIG. 25C shows the configuration for recording and reproducing theprogram on and from the flexible disk FD.

FIG. 26 is a block diagram showing the overall configuration of acontent providing system ex100 for realizing content distributionservice.

FIG. 27 is a diagram showing a mobile phone ex115 which uses the movingpicture coding method and the moving picture decoding method asexplained in the embodiments.

FIG. 28 is a block diagram showing the configuration of a mobile phone.

FIG. 29 is an illustration showing an example of a digital broadcastingsystem.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, the embodiments of the present invention will be explainedwith reference to FIG. 5 to FIG. 29.

First Embodiment

FIG. 5 is a block diagram showing the configuration of a moving picturecoding apparatus 100 in the first embodiment of the present invention.The moving picture coding apparatus 100 is a moving picture codingapparatus that, during the coding of each picture, only stores in memorythe motion vectors estimated from a reference picture coded immediatelybefore a current picture to be coded and, when during the coding of aB-picture in direct mode, the vector to be referred is not stored withinthe memory, codes using a different method from an existing direct mode.The apparatus includes a frame memory 101, a difference calculation unit102, a prediction error coding unit 103, a bit stream generation unit104, a prediction error decoding unit 105, an addition unit 106, a framememory 107, a motion vector estimation unit 108, a mode selection unit109, a coding control unit 110, switches 111 to 115, and a motion vectorstorage unit 116. The frame memory 101 is a picture memory for holdinginputted pictures on a per-picture basis, and provides a storage areawhere pictures inputted in display order are reordered into codingorder. The difference calculation unit 102 calculates for output, aprediction error, which is the difference between an image read out on aper-block basis for a picture within the frame memory 101 and, an imagefrom one block read out based on a motion vector from a referencepicture within the frame memory 107. The prediction error coding unit103 performs a frequency transformation and quantization on theprediction error obtained by the difference calculation unit 102, foroutput. In this quantization by the prediction error coding unit 103,cases wherein processes that become irreversible due to rounding off ofdivision results, and the like, are included. After converting thequantized result from prediction error coding unit 103 into variablelength code, the bit stream generation unit 104 converts the variablelength code into the coded bit stream format for output, attaching addedinformation such as header information which describes the relatedinformation of the coded prediction error, and generates a bit stream.The prediction error decoding unit 105 undertakes a variable lengthdecoding of the coded result from the prediction error coding unit 103.Then, after inverse quantization, the prediction error decoding unit 105performs an inverse frequency transformation and outputs the decodedprediction error. The addition unit 106 adds the block of theaforementioned reference image to the block of the prediction errorwhich is the decoded result from the prediction error decoding unit 105,then outputs the image for one decoded block. The frame memory 107 is apicture memory for accumulating the images of respective decoded blocks,and for storing reference images on a per-picture basis.

The motion vector estimation unit 108 estimates a motion vector of acurrent picture by blocks of predetermined size. The mode selection unit109 uses the current block read out from the frame memory 101 and themotion vector estimated for the current block as reference, and selectswhether to calculate the motion vectors of a current block by directmode or by another mode. In the case where direct mode is chosen, themode selection unit 109 calculates the motion vector of the currentblock using the motion vector of the reference picture subsequent indisplay order to the current picture to be coded, stored in the motionvector storage unit 116. The mode selection unit 109 reads out from theframe memory 107, the block of the reference image indicated by themotion vector, which is the result of the calculation, and outputs thisto the difference calculation unit 102 and the addition unit 106. Thecoding control unit 110 rearranges the inputted pictures stored indisplay order within the frame memory 101 into coding order, and, readsout the block, which becomes the unit for the coding process, in codingorder starting from the current picture to be coded. Moreover, codingcontrol unit 110 assigns reference indices to each reference picture andmanages the reference index list, which is a listing of the referenceindices assigned. In addition, the coding control unit 110 stores intothe motion vector storage unit 116, only the motion vector of thereference picture coded immediately before the current picture to becoded, then creates information indicating the storage size of themotion vectors stored in the motion vector storage unit 116. The motionvector storage unit 116 provides a storage area for the storage of themotion vector obtained from reference picture coded immediately beforethe current picture to be coded.

From hereon, the operation of the moving picture coding apparatus 100,as configured above, shall be explained. An input picture for the movingpicture coding apparatus 100, shown in FIG. 5, is inputted into theframe memory 101 in display order, on a per-picture basis. FIG. 6A is adiagram showing the sequence of pictures inputted as input pictures intothe moving picture coding apparatus 100 in FIG. 5. The sequence ofpictures inputted into the frame memory 101 is indicated in FIG. 6A. InFIG. 6A, vertical lines indicate pictures, and an alphabet and a numberat the lower right of each picture indicate a picture type (I, P or B)and a picture number in display order. Each picture inputted into framememory 101 shall be reordered into coding order. The reordering intocoding order is performed based on the reference relations in interpicture predictive coding, that is, with reordering done so that thepicture used as a reference picture is coded before the current pictureto be coded. FIG. 6B is a diagram showing the sequence of pictures whenthe sequence of pictures indicated in FIG. 6A is reordered into codingorder. For example, a P-picture shall use as reference, any one picturefrom among three neighboring I-pictures, or P-pictures, that precede indisplay order. In other words, each block to be coded within the currentP-picture to be coded shall refer to a single reference pictureappearing ahead in display order, with a motion compensation beingperformed. And, for each block, this single picture can be selectedarbitrarily from among three neighboring I-pictures, or P-pictures,preceding in display order. In addition, a B-picture shall use asreference pictures three nearby I-pictures, or P-pictures, that precedein display order, and one nearby I-picture, or P-picture appearingsubsequently in display order. Accordingly, for each block to be codedwithin the current B-picture to be coded, motion compensation isperformed in reference to two reference pictures. And, for every block,these two pictures can be arbitrarily selected from among three nearbyI-pictures, or P-pictures, that precede display order, and one nearbyI-picture, or P-picture appearing subsequently in display order. Assuch, it becomes necessary to code the single nearby I-picture, orP-picture, subsequent to the current B-picture in display order, at apoint prior to coding a B-picture. In this case, the result ofreordering the pictures in FIG. 6A shall be like that shown in FIG. 6B.Taking a B-picture, a picture B5, as an example, the P-picture (orI-picture) subsequent to the picture B5 in display order, which is alsothe closest picture to the picture B5, the picture P7, shall be codedprior to the coding of the picture B5.

It is assumed that the pictures reordered in the frame memory 101 areread out on a macroblock basis. Here, a macroblock is 16 horizontal×16vertical pixels in size. It is also assumed that motion compensation isperformed on a per-block basis (set herein as 8 pixels×8 pixels insize). From this point, the coding process for the picture P13 and thepicture B11 shall be discussed in sequence. Moreover, it is also assumedthat in the present embodiment, the coding control unit 110 manages thereference indices.

(Coding Process for Picture P13)

Since the picture P13 is a P-picture, inter picture predictive coding isperformed using forward reference. The reference pictures in this caseare the picture P10, the picture P7, and the picture P4. These referencepictures are already coded, and the decoded pictures thereof are storedin the frame memory 107.

The coding control unit 110 determines the picture type (I, P or B) tobe used for coding an inputted picture, and controls the switches 113,114 and 115 depending on the picture type. Here, a picture type isgenerally determined using a method for assigning a picture typeperiodically, for example. The reordering of pictures into coding orderwithin the frame memory 101 is carried out based on the picture typedetermined.

When coding a P-picture, the coding control unit 110 controls theswitches 113, 114 and 115 to be ON. Therefore, macroblocks in thepicture P13 which are read out from the frame memory 101 are inputtedinto the motion vector estimation unit 108, the mode selection unit 109and the difference calculation unit 102.

The motion vector estimation unit 108 estimates the motion vector ofeach block in the macroblock using the decoded image data of the pictureP10 stored in the frame memory 107 as a reference picture, and outputsthe estimated motion vectors to the mode selection unit 109.

The mode selection unit 109 determines the coding mode of the macroblockusing the motion vectors estimated by the motion vector estimation unit108. Here, coding mode refers to the method in which a macroblock is tobe coded. For example, in the case of a P-picture, the coding method tobe used can be determined from among the following: intra picturecoding, inter picture predictive coding using motion vectors, and interpicture predictive coding without the use of motion vectors (treating amotion vector as “0”, or selecting a motion vector from surroundingblocks). In determining a coding mode, the method with the least codingerrors within a small bit size is generally selected.

The coding mode determined by the mode selection unit 109 is outputtedto the bit stream generation unit 104. In addition, in the case wherethe current picture is the picture used as a reference picture duringthe coding of another picture and, the coding mode determined by themode selection unit 109 is an inter picture predictive coding mode, themode selection unit 109 stores the motion vector to be used in thatinter picture predictive coding into the motion vector storage unit 116.However, the motion vector is stored into the motion vector storage unit116 under the control of the coding control unit 110, as describedlater. Moreover, a motion vector is outputted from the mode selectionunit 109 into the bit stream generation unit 104.

Hereinafter, the storage operation of a motion vector from modeselection unit 109 into the motion vector 116 shall be explained. Thenumber of motion vectors stored in motion vector storage unit 116 shallbe designated by the coding control unit 110. A P-picture is coded usingthree preceding pictures as reference pictures. However, here, themotion vectors of a number of pictures which are fewer in number thanthe number of reference pictures, are stored in the motion vectorstorage unit 116. For example, it is assumed here that the motion vectorof only one picture is stored. In this case, with regard to the motionvector to be stored, a method of storing exists wherein, among referencepictures, the motion vector of the reference picture coded just beforethe current picture in coding order is stored. Here, motion vectorsalready stored in the motion vector storage unit 116 may be deleted uponthe start of coding of the current picture. And, it is also possible tooverwrite the motion vectors, in the order of blocks to be coded, at thetime a coding method is determined. Furthermore, the amount of motionvectors to be stored may be determined based on the size of the area(memory) contained in the decoding apparatus, in which the motionvectors are stored. As to the motion vectors to be stored in the motionvector storage unit 116, it may be done in such a way that the motionvectors of I-pictures only, P-pictures only, or I-pictures andP-pictures only will be stored. In addition to these, in a picture beingcoded, the motion vectors of a predetermined number of blocks, or of allblocks, coded ahead of the current block using an inter pictureprediction other than direct mode, may be stored. Here, information onmotion vectors to be stored with regard to an I-picture refers toinformation indicating that a motion vector is not stored (as I-picturesare not subject to motion compensation). And, as to motion vectors to bestored in the motion vector storage unit 116, whether, a few pictures,starting from the picture closest to the current picture to be coded indisplay order are stored, or a few pictures, starting from the pictureclosest (from a picture coming subsequently, in stream sequence) to thecurrent picture to be coded in coding order (stream sequence), arestored, either case is acceptable. In the case where a B-picture refersto two subsequent P-pictures, since the pictures are temporally apartalthough close in coding (or decoding) order, there is a highpossibility that as to picture content, they may not be considered asbeing close (not suitable for motion compensation). As such, in the casewhere a B-picture refers to a plurality of subsequent P-pictures, thereare cases where storing motion vectors, starting from the motion vectorclosest to the current picture to be coded in display order, isadvantageous. On the other hand, there is the merit of being able tosimplify memory management when storing motion vectors, from the motionvector of the picture closest to the current picture (from a subsequentpicture in stream sequence), in coding order (stream sequence).

Furthermore, the number of motion vectors (for example, the number ofreference pictures, the number of macroblocks) that are stored in themotion vector storage unit 116, as designated by the coding control unit110, is outputted into the bit stream generation unit 104.

Next, as shown in FIG. 5, the reference image based on the coding modedetermined by the mode selection unit 109 is outputted to the differencecalculation unit 102 and to the addition unit 106. However, in the casewhere intra picture coding is selected, a reference image is notoutputted. In addition, the coding control unit 110 controls the switch111 to switch to a terminal “a”, and the switch 112 to terminal “c”, inthe case where intra picture coding is selected by the mode control unit109. And, in the case where inter picture predictive coding is selected,the coding control unit 110 controls the switch 111 to connect to aterminal “b” and the switch 112 to terminal “d”.

Hereinafter, the case where inter picture predictive coding is selectedby the mode selection unit 109 shall be explained.

Apart from macroblock image data, reference images from the modeselection unit 109 are inputted into the difference calculation unit102. The difference calculation unit 102 calculates the differencebetween the macroblock image data and a reference image, and generates aprediction error image for output.

The prediction error image is inputted to the prediction error codingunit 103. The prediction error coding unit 103 performs frequencytransformation, quantization, and other coding processes on theprediction error image, from which coded data is generated for output.Here, frequency transformation and quantization processes can be done inunits of 8 horizontal×8 vertical pixels, for example. The coded dataobtained from the prediction error coding unit 103 is inputted to thebit stream generation unit 104 and the prediction error decoding unit105.

The bit stream generation unit 104 performs variable length coding, andso on, on the inputted coded data. It then generates a bit stream byadding the motion vector information, coding mode information, otherheader information, and so on, obtained from the mode selection unit109. In this case, information concerning the storage size of motionvectors determined by the coding control unit 110 (i.e., the number ofreference pictures, the number of macroblocks) is described within theheader information. This description should also be placed in a headerof the entire bit stream (sequence header), a header for a plurality ofpictures forming a “group of pictures” (GOP header), a picture header, aslice header, and others. And, in the case where the informationregarding the storage size of motion vectors is already shared by thecoding apparatus and the decoding apparatus, the information regardingthe storage size of a motion vector may or may not be coded. FIG. 7A to7D are diagrams showing an example of a bit stream generated by the bitstream generation unit 104 shown in FIG. 5. In the following diagrams,each shaded area represents a header. FIG. 7A is a diagram showing anexample of a bit stream in which information regarding the storage sizeof motion vectors is described in a sequence header. FIG. 7B is adiagram showing an example of a bit stream in which informationregarding the storage size of motion vectors is described in a GOPheader. FIG. 7C is a diagram showing an example of a bit stream in whichinformation regarding the storage size of motion vectors is described ina picture header. FIG. 7D is a diagram showing an example of a bitstream in which information regarding the storage size of motion vectorsis described in a slice header. Furthermore, it is possible to describethis information only once in the header of the entire bit stream. And,it is also possible to describe the same information repeatedly, intothe header for GOP (GOP header), the picture header, as well as theslice header, and others.

Moreover, the aforementioned information regarding the storage size ofmotion vectors shall not be limited to the number of reference picturesor the number of macroblocks. For example, it may simply be informationindicating a limitation set on the storage size of a motion vector, orinformation indicating a limitation on data to be stored into a memory,and so on.

The prediction error decoding unit 105 performs inverse quantization,inverse frequency transformation, and other decoding processes on codeddata inputted, and generates a decoded error image for output. Thedecoded error image is added to a predictive image by the addition unit106, to form a decoded image to be stored in the frame memory 107.

The coding for the remaining macroblocks of the picture 13 shall beperformed under the same process. Then, after processing of all themacroblocks of the picture P13 is completed, the coding process for thepicture B11 shall be performed.

(Coding of Picture B11)

Since the picture B11 is a B-picture, inter picture predictive codingusing bi-prediction reference shall be performed. In this case, thereference pictures shall be the picture P10, the picture P7, and thepicture P4 that precede the picture B11 in display order, and thepicture P13 appearing after the picture B11 in display order. Whencoding the picture B11, the coding control unit 110 controls the switch113 to be ON and the switches 114 and 115 to be OFF. Therefore, themacroblocks in the picture B11 which are read out from the frame memory101 are inputted to the motion vector estimation unit 108, the modeselection unit 109 and the difference calculation unit 102.

The motion vector estimation unit 108 estimates the forward motionvectors and the backward motion vectors of each block in a macroblockusing the decoded picture data of the picture P10, the picture P7, andthe picture P4 stored in the frame memory 107 as forward referencepictures and the decoded picture data of the picture P13 as a backwardreference picture.

The mode selection unit 109 determines a coding mode of the macroblockusing the motion vectors estimated by the motion vector estimation unit108. Here, a coding mode for a B-picture may be selected from codingmodes such as, intra picture coding, inter picture predictive codingusing forward motion vectors, inter picture predictive coding usingbackward motion vectors, inter picture predictive coding usingbi-predictive motion vectors, or direct mode, for example.

Here, the case where coding by direct mode shall be explained. FIG. 8 isa diagram showing an example of a calculation method in the case where amotion vector of a current block is calculated using direct mode. InFIG. 8, it is assumed that the current block to be coded is a block “a”within a picture B11. The motion vector of a block, co-located with theblock “a”, within the reference picture whose second reference index is“0” for the picture B11, shall be used in coding the block “a” in directmode.

First, the case where the reference picture with a second referenceindex as “0” for the picture B-11 is a picture P13 shall be explained.In order to identify which picture a particular motion vector belongsto, the motion vectors stored in motion vector storage unit 116 arestored with correspondence to picture numbers. The mode selection unit109 searches for the picture number “13” corresponding to the referencepicture with the second reference index as “0” for picture B11, withinthe reference index list of the picture B11. It then determines whetherthe motion vectors of the reference picture of the picture number “13”is stored within the motion vector storage unit 116 or not. As thepicture P13 is the reference picture coded closest to the picture B11 incoding order, the motion vectors used for the coding of each block ofthe picture P13, are stored in the motion vector storage unit 116. Assuch, when the block “a” is coded in direct mode, the motion vector of ablock “b” within the picture P13 shall be used. A motion vector “c” isthe motion vector used during the coding of the block “b”, and is storedin the motion vector storage unit 116. The motion vector “c” refers tothe picture P10. For the block “a”, a bi-prediction is performed fromthe picture P10 and the picture P13, using the motion vectors generatedfrom the motion vector “c”. The motion vectors used during the coding ofthe block “a”, in this case are, a motion vector “d” from the pictureP10, and a motion vector “e” from the picture P13.

Here, the first method in generating the motion vector for the block “a”from the motion vector “c” is the method of scaling using the temporaldistance between pictures. Now, if we assume the size of the forwardmotion vector “d” is MVF, the size of the backward motion vector “e” isMVB, the size of the motion vector “c” is MVP, the temporal distancebetween the reference picture (the picture P13) of the current picture(the picture B11) with a second reference index as “0”, and the picture(the picture P10) used as reference by the block of the said referencepicture (the picture P13) is TRD, and the temporal distance between thecurrent picture (the picture B11) and the picture (the picture P10) usedas reference by the block of the reference picture (the picture P13) isTRB, the motion vector MVF and the motion vector MVB are calculated byEquation 1 and Equation 2, respectively.MVF=MVP×TRB/TRD  (Equation 1)MVB=(TRB−TRD)×MV/TRD  (Equation 2)where MVF, MVB respectively represent a combination of the horizontalcomponents and vertical components of the motion vectors.

Furthermore, the second method for generating the motion vector of theblock “a” from the motion vector “c” is the method where scaling iscarried out on the motion vector “c” using a predetermined scalingcoefficient. The scaling coefficient to be used here may be adjusted ona multi-block basis or a multi-picture basis. Moreover, this scalingcoefficient is determined uniquely in a system and may be shared by thecoding side and the decoding side, and, by description as headerinformation within a bit stream, may also be shared by the coding sideand decoding side.

Next, the case wherein the reference picture of a second reference indexas “0” is a reference picture other than the picture P13 shall beexplained. For example, it is possible to have a case where any oneamong the pictures, the picture P10, the picture P7, and the picture P4as being the reference picture of the second reference index having avalue as “0” for the picture B11. For reference pictures other than thepicture P13, the motion vectors used during the coding of each block forthese pictures are not stored into the motion vector storage unit 116,because in coding order, such pictures are not reference pictures codedclosest to the picture B 11. As such, even if the block “a” shall becoded in direct mode, the motion vectors of the reference picture of thesecond reference index with a value “0” cannot be used.

The first processing method in direct mode for this case is abi-prediction method with the motion vector as “0”. As referencepictures in this case, for example, for the picture B11, the picturewith a first reference picture as “0” and the picture with a secondreference index as “0” can be used. FIG. 9 is a diagram showing anexample of a bi-prediction in the case where, during the coding of aB-picture in direct mode, the motion vector of the reference picturewith a second reference index as “0” cannot be used. For example, in thecase where the picture with a first reference index as “0” is a pictureP7, and the picture with a second reference index as “0” is a pictureP10, a bi-prediction can be performed for a block “a”, as shown in FIG.9. Motion compensation shall be performed on the block “a”, referring toco-located blocks within the picture P7 and the picture P10.

In addition, the second processing method in direct mode is a methodusing the motion vectors of surrounding blocks within the same picture.However, in this case, it is assumed that the motion vectors of theblocks surrounding the current block are stored by the mode selectionunit 109 into the motion vector storage unit 116. In this method, asshown in FIG. 10, in the case where a block BL is coded in direct mode,the motion vectors used during the coding of three surrounding blocks, ablock BL1, a block BL2, and a block BL3, shall be used to determine themotion vector to be used for the coding of the block BL. In determiningthe motion vectors, a method that calculates the median or average ofthe motion vectors of the three surrounding blocks, with regard toforward motion vectors and backward motion vectors respectively, can beused, for example. Likewise, a method that calculates the median oraverage of the motion vectors of the three surrounding blocks, withregard to the motion vector referring to the first reference picture andthe motion vector referring to the second reference picturerespectively, can also be used.

Now, as shown in FIG. 5, the coding mode determined by the modeselection unit 109 is outputted to the bit stream generation unit 104.In addition, the reference image based on the coding mode determined bythe mode selection unit 109 is outputted to the difference calculationunit 102 and the addition unit 106. However, in the case where intrapicture coding is selected, a reference image shall not be outputted.Moreover, in the case where intra picture coding is selected by the modeselection unit 109, the coding control unit 110 controls the switch 111to connect to the terminal “a”, and the switch 112 to the terminal “c”.In the case where inter picture predictive coding is selected, thecoding control unit 110 controls the switch 111 to connect to theterminal “b”, and the switch 112 to the terminal “d”. The case were themode selection unit 109 has selected inter picture prediction shall beexplained below.

A reference image is inputted from the mode selection unit 109 into thedifference calculation unit 102. The difference calculation unit 102calculates the difference between a macroblock of picture B11 and thereference image, and generates a prediction error image for output.

The prediction error image is inputted to the prediction error codingunit 103. Frequency transformation, quantization and other codingprocesses are performed on the inputted prediction error image by theprediction error coding unit 103 and coded data is generated for output.The coded data outputted by the prediction error coding unit 103 isinputted to the bit stream generation unit 104.

The bit stream generation unit 104 performs variable length coding, andso on, on the inputted coded data and, adding information obtained frommode selection unit 109 such as, motion vector information, coding modeinformation, other header information, and so on, generates a bitstream. However, with regard to macroblocks coded in direct mode, motionvector information is not added to the bit stream.

The remaining macroblocks of the picture B11 are coded through the sameprocess. And, when processing for all the macroblocks of the picture B11are completed, the coding process for the picture B-12 shall beperformed.

As discussed so far, in the moving picture coding method in the presentinvention, a limitation is set on the number (i.e., how manypictures-worth) of motion vectors to be stored with regard to motionvectors used during coding in direct mode. And, the information(limitation information, as well as storage information as definedwithin the claims) indicating the addition of this limitation shall bedescribed as header information within a bit stream. Moreover, duringcoding in the direct mode, in the case where the motion vector to beused shall be stored in a memory, the said motion vector shall be used.On the other hand, in the case where the motion vector to be used shallnot be stored in memory, coding can be performed, with the motion vectorof the current block as “0”, or by using the motion vectors used duringthe coding of previously coded blocks surrounding the current block,within the same picture.

Through processes such as this, the storage size of motion vectors thatneed to be stored for coding in direct mode is reduced, and reduction ofhardware and software memory size becomes possible. Furthermore, even inthe case where a motion vector to be used in coding in direct mode isnot available, it is possible to perform coding in direct mode using analternative method. And, as explained above, since the present inventioncan be implemented regardless of the picture with a second referenceindex as “0”, there is no need to redefine the setting for a picturewith a second reference index as “0”, and design complications can beavoided.

Furthermore, although the present embodiment has been explained on theassumption that the coding process for macroblocks is performed in unitsof 16 horizontal×16 vertical pixels, motion compensation in blocks of 8horizontal×8 vertical pixels, and the coding of predictive error imagesof blocks in units of 8 horizontal×8 vertical pixels, the units for thesaid processes may be of a different pixel size.

Moreover, although in the explanation using FIG. 10 in the presentembodiment, the method of determining the motion vector to be used indirect mode where the motion vectors of three previously codedsurrounding blocks are to be used is explained, a different value andmethod may be used as to the number of the said surrounding blocks, andthe method of determining the prediction value. For example, a method,using as prediction value, the motion vector of a block locatedadjacently to the left of the current block, as well as other methodsare available.

Note that in the present embodiment, the case where a P-picture uses asreference, any one picture from among three neighboring I-pictures, orP-pictures, which appear ahead in display order is explained. Likewise,the case where a B-picture uses any one picture from among threeneighboring I-pictures, or P-pictures, which appear ahead in displayorder as a forward reference picture, and a single neighboring I-pictureor P-picture appearing subsequently in display order, as a backwardreference picture, is also explained. However, it is possible for thenumber of such reference pictures to be different.

Also, although the case wherein a B-picture is not used as a referencepicture is explained in the present embodiment, a case where a B-pictureis used as a reference picture is possible. In the case where aB-picture is used as a reference picture, since there is a need to storethe resultant picture from the local decoding of a B-picture, as areference picture, into the frame memory 107, the controlling of theswitch 114 and the switch 115 by the coding control unit 110 shall bethe same as in the case of the coding of a P-picture in the presentembodiment. And, even in the case where a B-picture is used as thereference picture, wherein the B-picture shall not be the picture withthe second reference index as “0” for other pictures, the motion vectorsused during the coding of the B-picture need not be stored into themotion vector storage unit 116. This is because, in the case where aB-picture cannot be the picture with a second reference index as “0”,the picture with a second reference index as “0” can only be either anI-picture or a P-picture. Accordingly, in this case, only the motionvectors used during the coding of an I-picture or P-picture need to bestored into the motion vector storage unit 116. In so doing, the motionvectors used in the coding of the picture with a second reference as “0”for the current picture to be coded in direct mode may be used, allowingan improvement in coding efficiency. Here, the decision as to whetherthe motion vectors used during the coding of a B-picture shall to bestored in the motion vector storage unit 116 or not, may bepre-determined in the coding apparatus-side and decoding apparatus-side.Likewise, the coding apparatus-side may also determine the storage anddescribe the information in the header section of the bit stream. In thecase where the storage is determined by the coding apparatus-side, thedetermination is done by the coding control unit 110, and theinformation may be described within a header by bit stream generationunit 104.

In addition, in pre-determining within the coding apparatus-side anddecoding apparatus-side, whether the motion vectors used during thecoding of a picture shall be stored in motion vector storage unit 116 ornot, it is possible not to have the bit stream generation unit 104describe such determination information within a header.

Furthermore, with regard to the motion vectors to be stored for directmode, the case where the motion vectors used during the coding of asingle reference picture coded immediately before the current picture incoding order is stored, is explained in the present embodiment. However,the storage of motion vectors of reference pictures, differing in numberthan as described here, is possible.

Moreover, it is possible to assign directly to the decoding apparatuswhich among the motion vectors shall be stored or not. For example, bydesignating the picture numbers of the pictures to which the storage ornon-storage of their motion vectors will be assigned, within the headerof the picture where the storage or non-storage of it's motion vectorshall be designated, or within the header of another picture, it ispossible to instruct the decoding apparatus as to which pictures' motionvectors are stored and which pictures' motion vectors are not stored.For example, in the case where the reference picture with a secondreference index as “0” during the coding of the picture B11 is known tobe the picture P10, instructions are given for the motion vectors of thepicture P10 to be stored, and for the motion vector of the picture P13not to be stored. As a result, the motion vector of the picture P10 (thepicture with a second reference index as “0”) is stored in the motionvector storage unit 116 and, it is possible to use the motion vector ofthe reference picture with a second reference index as “0” in directmode. Here, it will be advantageous to describe the informationdetermining which motion vectors are to be retained, as headerinformation within a bit stream. One example is to describe (i.e., as“0”, or “1”) the information indicating whether the motion vectors foreach picture shall be stored or not, within a header in a bit stream. Adifferent example would be to describe (i.e., if not for storage “0”, oras accompanying information on a particular picture “3”, in a caseindicating the need to re-store on the third picture following theparticular picture) the information indicating the period for storingmotion vectors. With the use of these methods, the flexible managementof motion vector storage becomes possible, allowing for improvements incoding efficiency.

Second Embodiment

A moving picture decoding apparatus 700 in the second embodiment of thepresent invention will be explained with reference to FIG. 11. FIG. 11is a block diagram showing the configuration of the moving picturedecoding apparatus 700 in the second embodiment. The moving picturedecoding apparatus 700 is a moving picture decoding apparatus thatstores the motion vectors of reference pictures into the memoryaccording to the motion vector storage method described as headerinformation within a bit stream, and decodes current pictures fordecoding. It includes a bit stream analysis unit 701, a prediction errordecoding unit 702, a mode decoding unit 703, a motion compensationdecoding unit 705, a motion vector storage unit 706, a frame memory 707,an addition unit 708, a switch 709 and a switch 710.

FIG. 12A is a diagram showing the sequence of pictures included within abit stream. FIG. 12B is a diagram showing the sequence of the picturesshown in FIG. 12A, when reordered into decoding order. The pictureswithin a bit stream shall be inputted into the moving picture decodingapparatus 700 in the sequence shown in FIG. 12A. Here it is assumed thata P-picture is coded using as reference, any one picture from amongthree neighboring I-pictures, or P-pictures, appearing ahead in displayorder, and that a B-picture is coded using any one picture from amongthree neighboring I-pictures, or P-pictures, appearing ahead in displayorder, and a single neighboring I-picture or P-picture appearingsubsequently in display order, as reference pictures. Hereinafter, thedecoding process of a picture P13 and a picture B11 shall be explainedin sequence, as cases in point in the decoding of a P-picture and aB-picture. Furthermore, it is assumed that the bit stream analysis unit701 manages reference indices in the present embodiment.

(Decoding of Picture P13)

Bit streams, being the input for the moving picture decoding apparatus700, are first inputted into the bit stream analysis unit 701. The bitstream analysis unit 701 extracts various data from the inputted bitstream. Here, various data includes information on coding mode, motionvector information indicating coded motion vectors, and so on. Theextracted coding mode information is outputted to the mode decoding unit703. The extracted motion vector information is outputted to the motioncompensation decoding unit 705. Further, the extracted coded predictionerror data is outputted to the prediction error decoding unit 702.

The mode decoding unit 703 controls the switch 709 and the switch 710with reference to the coding mode information extracted from the bitstream. If the coding mode selected is intra picture coding, the modedecoding unit 703 controls the switch 709 to connect to a terminal “a”and the switch 710 to a terminal “c”, and if the coding mode is interpicture predictive coding, it controls the switch 709 to connect to aterminal “b” and the switch 710 to a terminal “d”.

In addition, the mode decoding unit 703 outputs the mode selectioninformation to the motion compensation decoding unit 705. Hereinafter,the case where the mode selected is inter picture predictive codingshall be explained.

The prediction error decoding unit 702 decodes the inputted codedprediction error data and generates a prediction error image. Theprediction error image generated is outputted to the switch 709. Here,as the switch 709 is connected to the terminal “b”, the prediction errorimage is outputted to the addition unit 708.

The motion compensation decoding unit 705 decodes the motion vectorinformation, which is the coded motion vector inputted from the bitstream analysis unit 701. And based on the decoded reference picturenumber and the motion vector, the motion compensation decoding unit 705obtains a motion compensation image (block) from the frame memory 707.The motion compensation image, which is generated in this manner, isoutputted to the addition unit 708.

Furthermore, in the case where the decoding of a picture used as areference picture during the decoding of another picture is performed,the motion compensation decoding unit 705 stores the motion vectors intothe motion vector storage unit 706. Here, since a P-picture shall beused as a reference picture, the motion vector obtained during thedecoding of the picture P13 shall be stored in the motion vector storageunit 706. However, the storage of motion vectors into the motion vectorstorage unit 706 is controlled based on the header information in thebit stream.

Now, the operation of the motion vector storage unit 706 shall beexplained. The number of motion vectors to be stored in the motionvector storage unit 706 is described within a header in a bit stream.This header information is extracted by the bit stream analysis unit 701and outputted to the motion compensation decoding unit 705. Here, it isassumed that it is described within the header that the motion vectorsof only one picture shall be stored. In this case, with regard to theamount of motion vectors to be stored, there is a method for storing themotion vectors of recently decoded reference pictures in coding order.Here, at the start of the decoding of the current picture, it ispossible to delete the motion vectors already stored in the motionvector storage unit 706, and each time a motion vector is obtainedduring the decoding of a block, the motion vectors can be overwritten insequence.

The inputted prediction error image and motion compensation image areadded by the addition unit 708, and a decoded image is generated. Thedecoded image generated is outputted to the frame memory 707 via theswitch 710.

As in the manner stated above, the macroblocks of the picture P13 aredecoded one by one. Once all the macroblocks of the picture P13 aredecoded, the decoding of picture B11 shall be performed.

(Decoding of Picture B11)

Since the operations of the bit stream analysis unit 701, the modedecoding unit 703, and the prediction error decoding unit 702 are thesame as in the decoding of picture P13, the explanation shall not berepeated.

The motion compensation decoding unit 705 decodes the coded motionvectors inputted from the bit stream analysis unit 701, then, obtains amotion compensation image (block) from the frame memory 707, based onthe decoded motion vectors.

Here, the case where the coding mode obtained by the mode decoding unit703 is a direct mode shall be explained. FIG. 8 is a diagram showing thepictures P10 to P13, as arranged in temporal order, where the currentblock to be decoded is assumed to be the block “a”. In the case wherethe block “a” shall be decoded in direct mode, the motion vector of theblock, co-located with the block “a”, within the reference picture witha second reference index as “0” shall be used.

First, the case where the reference picture, with a second referenceindex as “0”, is the picture P13 shall be explained. As shown above inthe decoding of the picture P13, here it is described within the headerthat the motion vectors of only one picture shall be stored. And, amongthe reference pictures, the motion vectors of the reference picturecoded immediately before the current picture in decoding order, isstored. Moreover, since in decoding order, the picture P13 is thereference picture decoded most immediately before the picture B11, themotion vectors obtained during the decoding of each block of the pictureP13 are stored in the motion vector storage unit 706. As such, duringthe decoding of the block “a” in direct mode, the motion vector of theblock “b” in the picture P13 shall be used. The motion vector “c” is themotion vector used during the decoding of the block “b”, and is storedin the motion vector storage unit 706. The motion vector “c” uses thepicture P10 as reference. For the block “a”, a bi-prediction isperformed from the reference pictures, the picture P10 and the pictureP13, using the motion vectors generated from the motion vector “c”. Themotion vectors used in this case of coding of the block “a” are themotion vector “d” from the picture P10, and the motion vector “e” fromthe picture P13.

Here, the first method in generating the motion vector of the block “a”from the motion vector “c” is the method of scaling using the temporaldistance between pictures. Now, if we assume the size of the forwardmotion vector “d” is MVF, the size of the backward motion vector “e” isMVB, the size of the motion vector “c” is MVP, the temporal distancebetween the reference picture (the picture P13) with a second referenceindex as “0” for the current picture (the picture B11), and the picture(the picture P10) used as reference by the block of said referencepicture with a second reference index as “0”, is TRD, and the temporaldistance between the current picture (the picture B11) and the picture(the picture P10) used as reference by the block of the referencepicture with a second reference index as “0”, is TRB, the motion vectorMVF and the motion vector MVB are respectively calculated by Equation 1and Equation 2.

In addition, the second method for generating the motion vector of theblock “a” from the motion vector “c” is the method where scaling iscarried out on the motion vector “c” using a predetermined scalingcoefficient. The scaling coefficient to be used here may be adjusted ona multi-block basis or a multi-picture basis. Moreover, this scalingcoefficient is determined uniquely in a system and may be shared by thecoding side and the decoding side. This scaling coefficient is describedas header information within a bit stream and may also be used uponextraction by the bit stream analysis unit 701.

Next, the case wherein the reference picture of the second referenceindex as “0” is a reference picture other than the picture P13 shall beexplained. For example, it is possible to have a case where any oneamong the pictures P10, the picture P7, and the picture P4 as being thereference picture of the second reference index having a value as “0”for the picture B11. For reference pictures other than the picture 13,the motion vectors used during the decoding of each block for thesepictures are not stored into the motion vector storage unit 706 becausein decoding order, such pictures are not reference pictures decoded mostimmediately before the picture B11 (in finding out which picture amotion vector stored in the motion vector storage unit 706 belongs to,or which picture the picture having a reference index as “0” is,respective picture numbers are obtained, and a decision is made bycomparing the respective picture numbers obtained). As such, during thedecoding of the block “a” in direct mode, the motion vectors of thereference picture of the second reference index with a value as “0”cannot be used.

The first processing method in direct mode for this case is abi-prediction method with the motion vector as “0”. For referencepictures in this case, for example, in terms of the picture B11, thepicture with a first reference picture as “0” and the picture with asecond reference index as “0” can be used. For example, in the casewhere the picture with a first reference index as “0” is the picture P7,and the picture with a second reference index as “0” is the picture P10,a bi-prediction can be performed for the block “a”, as shown in FIG. 9.

In addition, the second processing method in direct mode is a methodusing the motion vectors of surrounding blocks within the same picture.In this case, as shown in FIG. 10, in the case where the block BL isdecoded in direct mode, a method is available wherein the motion vectorsused during the decoding of three surrounding blocks, the block BL1, theblock BL2, and the block BL3, shall be used to determine the motionvector to be used during the decoding of the block BL. Here, as a methodfor determining the motion vector to be used in decoding, for example, amethod of calculating the median or average of the motion vectors of thethree surrounding blocks for each forward motion vector and backwardmotion vector respectively, is available, among others. Moreover, it isassumed that the motion vectors of the surrounding blocks in this caseare stored by the motion compensation decoding unit 705 into the motionvector storage unit 706. The motion compensation image, generated inthis manner, is outputted to the addition unit 708.

The inputted prediction error image and motion compensation image areadded by the addition unit 708, and a decoded image is generated. Thedecoded image generated is outputted to the frame memory 707 via theswitch 710.

As in the manner stated above, the macroblocks of the picture B11 aredecoded one by one. Once all the macroblocks of the picture B11 aredecoded, the decoding of picture B12 shall be performed. Furthermore,the decoded pictures are sequentially outputted from the frame memory707 as output pictures, as shown in FIG. 12B.

As explained so far, in the moving picture decoding method in thepresent invention, a bit stream is inputted with regard to the motionvectors to be used during coding in direct mode, said bit stream beingcoded with a limitation set on the number (i.e., how manypictures-worth) of motion vectors to be stored. This bit stream is thendecoded. During this decoding, the information indicating the additionof a limitation is extracted from the header information of the bitstream, and decoding is carried out based on the extracted information.In addition, during decoding in direct mode, in the case where themotion vector to be used shall be stored into the memory, such motionvector shall be used. And, in the case where the motion vector to beused shall not be stored into the memory, decoding is performed witheither the motion vector being treated as 0, or by using the motionvectors used during the decoding of the surrounding blocks within thesame picture.

Through these procedures, the bit stream coded with a limitation set onthe number of motion vectors to be used during coding in direct mode,can be decoded accurately. Also, the storage size of the vectors to bestored for direct mode, as well as hardware and software memory size canbe reduced. Furthermore, even in the case where a motion vector to beused in direct mode is not available, direct mode may be implementedusing an alternative method. And, as explained above, since the presentinvention can be implemented regardless of the picture with a secondreference index as “0”, there is no need to redefine the setting for apicture with a second reference index as “0”, and design complicationscan be avoided.

Moreover, although in the explanation using FIG. 10 in the presentembodiment, the method of determining the motion vector to be used indirect mode where the motion vectors of three previously decodedsurrounding blocks are to be used is explained, a different value andmethod may be used as to the number of the said surrounding blocks, andthe method of determining the prediction value. For example, a method,using as prediction value, the motion vector of a block locatedadjacently to the left of the current block, as well as other methods,are available.

In addition, although in the present embodiment, the case wherein aP-picture uses as reference, any one picture from among threeneighboring I-pictures, or P-pictures, which appear ahead in displayorder, as well as the case wherein a B-picture uses any one picture fromamong three neighboring I-pictures, or P-pictures, which appear ahead indisplay order as a forward reference picture, and a single neighboringI-picture or P-picture appearing subsequently in display order, as abackward reference picture, are explained, it is possible for the numberof such reference pictures to be different. Moreover, it is alsopossible to have a B-picture being used as a reference picture.

Also, although the case wherein a B-picture is not used as a referencepicture is explained in the present embodiment, a case where a B-pictureis used as a reference picture is possible. In the case where aB-picture is used as a reference picture, the resultant picture from thedecoding of a B-picture shall be stored as a reference picture in theframe memory 707. Furthermore, even in the case where a B-picture isused as a reference picture, the motion vectors used during the decodingof a B-picture need not be stored in the motion vector storage unit 706.Whether the motion vectors used during the decoding of a B-picture shallto be stored in the motion vector storage unit 706 or not, may bepre-determined in the coding apparatus-side and decoding apparatus-side,or determined in the coding apparatus-side, where it is also possible todetermine this decision through the extraction of information describedin the header section of the bit stream.

Furthermore, in the present embodiment, and with regard to the motionvectors to be stored for direct mode, the case where the motion vectorused during the decoding of a single reference picture decodedimmediately before the current picture in decoding order, is stored isexplained. However, the storage of motion vectors of a different numberof reference pictures is possible.

Moreover, it is possible to directly assign which ones, among motionvectors, are to be stored or not. For example, by designating thepicture numbers of the pictures to which the storage or non-storage oftheir motion vectors will be assigned, within the header of the picturewhere the storage or non-storage of it's motion vector shall bedesignated, or within the header of another picture, which pictures'motion vectors shall be stored and which pictures' motion vectors shallnot be stored are assumed to be designated within the header.Accordingly, such instruction information may be extracted from theheader information within a bit stream. Now, in the case where it isinstructed that the motion vector of the picture P10 is to be stored andthe motion vector of the picture P13 is not to be stored, the motionvectors of the picture P10 shall be stored into the motion vectorstorage unit 706, at the point of decoding of the picture B11. Here, inthe decoding of the picture B11, assuming that the reference picturewith a second reference index as “0” is the picture P10, it is possibleto use the motion vectors used during the decoding of the referencepicture with a second reference index as “0” in the direct mode. Withthe use of such methods, the more flexible management of motion vectorstorage becomes possible.

Moreover, it is possible to store the motion vectors used during thedecoding, by the decoding apparatus, of a P-picture coded by the codingapparatus immediately before the current picture in stream order,namely, in coding order, from among reference pictures referred to bythe picture containing the current block to be decoded. However, asmentioned above, it is also possible to store the motion vectors usedduring the decoding of the P-picture immediately preceding or aP-picture preceding the current picture in display order, amongreference pictures referred to by the picture containing the currentblock to be decoded. The decoded pictures shall be outputtedsequentially from the frame memory 707 as output pictures, after beingreordered into display order, based on the times of the pictures storedin the frame memory 707. In addition, the time carried by the respectivepictures may be obtained, for example, from the time informationoutputted by a system for transmitting the pictures via a transmissionchannel for packets, or the like, or from the time information withinthe format for multiplexing video streams and audio streams, or from thetime information within a video stream format. Moreover, such timeinformation may either be the absolute time identifying the time of eachpicture or the relative time identifying the temporal relation betweenpictures.

In this way, the motion vector used during the decoding of P-pictureimmediately preceding or a P-picture preceding the current picture indisplay order from among reference pictures referred to by a picturecontaining the current block to be decoded shall be stored and, bystoring the motion vector used during the decoding of a referencepicture temporally near the picture containing the current block to bedecoded, the possibility of improving the accuracy of motioncompensation is increased.

Third Embodiment

In the aforementioned first embodiment, the case where the informationregarding the storage size of motion vectors is described in a headerwithin a bit stream is discussed. However, the description of suchinformation should not be limited to the header within a bit stream. Forexample, such information may be described in the managementinformation, not included within a bit stream. Hereinafter, the movingpicture coding apparatus, as well as the recording apparatus in thethird embodiment of this present invention shall be explained, withreference to FIG. 13 to FIG. 17. FIG. 13 is a block diagram showing theoutline configuration of a recording apparatus 500, for recording onto arecording medium such as DVD, the management information for managingthe reproduction of data from the recording medium and coded AudioVisual (AV) data. The recording apparatus 500 is a recording apparatusthat codes and records video data accompanied by sound onto a recordingmedium, and includes a video coding unit 501, an audio coding unit 502,as well as a system coding 503. The video coding unit 501 is a movingpicture coding apparatus for coding inputted video data. The audiocoding unit 502 is an audio coding apparatus for coding inputted audiodata. The system coding unit 503 multiplexes a video stream and an audiostream, to generate an AV stream which is recorded on a recordingmedium. In addition, after recording the AV stream, the system codingunit 503, following input from an input unit, or the like, not shown inthe diagram, generates management information, and records themanagement information generated onto the recording medium, on an areaother than where the AV stream is recorded.

Hereinafter, the operation of the recording apparatus 500, as configuredabove, shall be explained. The video data inputted into the recordingapparatus 500 is inputted into the video coding unit 501, and theinputted audio data is inputted into the audio coding unit 502. Thevideo data inputted into the video coding unit 501 and the audio datainputted into the audio coding unit 502 are coded in real-time and insynchronization between the video coding unit 501 and the audio codingunit 502. The bit stream generated from the video coding unit 501 isoutputted, as a video stream, to the system coding unit 503. The bitstream coded by the audio coding unit 502 is outputted, as an audiostream, to the system coding unit 503. The system coding unit 503multiplexes the video stream inputted from the video coding unit 501 andthe audio stream inputted from the audio coding unit 502 and generatesan AV stream. The AV stream generated is recorded, in real-time, onto arecording medium. When the recording of the AV stream onto the recordingmedium is completed, the system coding unit 503 reads out the controlcodes generated by the video coding unit 501 and generates managementinformation which includes the control codes read out. The system codingunit 503 then records this management information onto the recordingmedium, on an area different from where the AV stream is recorded. FIG.14 is an illustration showing an example of a recording medium 400 suchas DVD, in which the management information and Audio Video (AV) streamsgenerated by the system coding unit 503, shown in FIG. 13, are recorded.As shown in FIG. 14, data is recorded concentrically within therecording medium 400, with the management information 401 and AV data402 being recorded on different areas within the recording medium 400.In general, the AV data 402 is recorded in an area towards the outerradius, and the management information 401, on an area towards the innerradius of the recording medium 400.

FIG. 15 is a block diagram showing the configuration of a moving picturecoding apparatus 900 included within the recording apparatus 500 shownin FIG. 13. The moving picture coding apparatus 900 corresponds to thevideo coding unit 501 indicated in FIG. 13. The moving picture codingapparatus 900 is a moving picture coding apparatus that outputs, on adifferent timing, and as different data from the bit stream mentionedearlier, control codes (control information as defined within theclaims) which indicate that a limitation is set on the data to be storedwithin the motion vector storage unit. The moving picture codingapparatus 900 includes a frame memory 101, a difference calculation unit102, a prediction error coding unit 103, a prediction error decodingunit 105, an addition unit 106, a frame memory 107, a motion vectorestimation unit 108, a mode selection unit 109, switches 111 to 115, amotion vector storage unit 116, a bit stream generation unit 904, aswell as a coding control unit 910. In the same diagram, since thestructural components that are the same as those in the moving picturecoding apparatus 100 shown in FIG. 5 are already explained, the samereference numbers shall be assigned and the explanations shall not berepeated. However, here, unlike in the first embodiment, during thecoding of a B-picture, the motion vector estimation unit 108 searchesmotion vectors using as reference four I-pictures, or P-pictures,appearing ahead of the current picture to be coded in display order, aswell as one I-picture, or P-picture appearing after the current pictureto be coded in display order. After performing variable length coding onthe quantized result from prediction error coding unit 103, the bitstream generation unit 904 converts the variable length codes into thecoded bit stream format for output, attaching added information such asheader information which describes the related information of the codedprediction error, and generates a bit stream. Furthermore, the bitstream generation unit 904 associates the control codes generated by thecoding control unit 910 with the bit streams and holds it. The bitstream generation unit 904 outputs the control codes on a differenttiming from the bit stream. More specifically, the control codes areoutputted according to external output instructions (i.e., the systemcoding unit 503 of the recording apparatus 500 shown in FIG. 13). Acontrol signal for synchronization is inputted into the control codingunit 910 from an external source, in order to code video data and audiodata in real time, in synchronization with the audio coding unit 502.The coding control unit 910 controls the mode selection unit 109 tostore in the motion vector storage unit 116 only the motion vectors ofthe reference picture appearing immediately after the current picture tobe coded in display order, in other words, the motion vectors of thereference picture coded immediately before the current picture to becoded. The coding control unit 109 then generates control codesindicating the setting of some limitation on the data to be storedwithin the memory. The coding control unit 910 may generate controlcodes at any timing, although generation at the start, or immediatelyafter, the coding process is desirable. In addition, the coding controlunit 910 may store the control codes internally, beforehand.

Hereinafter, the storage of reference pictures, as well as motionvectors into the memory by the moving picture coding apparatus 900, asconfigured above, shall be explained in comparison with the existingMPEG-4 coding format. FIG. 16A is a diagram showing pictures inputted indisplay order, and their respective coding order. FIG. 16B is a diagramshowing the reference pictures and motion vectors stored into the memoryfollowing the coding of each picture in the existing MPEG-4 format. FIG.17A is a diagram showing pictures inputted in display order, and theirrespective coding order. FIG. 17B is a diagram showing the referencepictures and motion vectors stored into the memory following the codingof each picture in the moving picture coding apparatus 900 shown in FIG.15. Here, in either case, it is assumed that B-pictures shall not bereferred to by other pictures, and that only I-pictures and P-picturesshall be referred to by other pictures. In other words, I-pictures andP-pictures may become the reference pictures of other pictures butB-pictures cannot become the reference pictures of other pictures.Moreover, FIG. 17A and FIG. 16A are identical diagrams.

In FIG. 17A and FIG. 16A, each picture included in the inputted picturesis represented by a vertical line. B-pictures, which will not bereferred to by other pictures, are represented by broken lines. Inaddition, the “B”, as well as “P”, assigned to each picture indicatesthe picture type, and the number on the right indicates the coding orderof each picture. In other words, as shown in FIG. 16A and FIG. 17A,pictures within the inputted pictures are inputted and displayed in thefollowing order: I0→B2→B3→P1→B5→B6→P4→B8→B9→P7→B11→B12→P10. In contrast,since a B-picture refers to a picture appearing subsequently in displayorder, before coding a B-picture, an I-picture or a P-picture appearingafter the (current) B-picture in display order, must be coded first.Accordingly, the pictures within the inputted pictures shall be coded inthe following order: I0→P1→B2→B3→P4→B5→B6 P7→B8→B9→P10→B11→B12. Assumingthat, in display order, a B-picture precedes the picture I0, after thecoding of the picture I0, such B-picture shall be coded prior to codingthe picture P1.

In the existing MPEG-4, the pixel values of the two reference picturescoded immediately before the current picture to be coded, and the motionvectors used for the motion compensation of such reference pictures,shall be stored within the memory. In general, as shown in FIG. 16B, thepixel values of the two reference pictures coded immediately before thecurrent picture to be coded, and the motion vectors of one referencepicture coded later, among the two reference pictures mentioned, arestored within the memory. In addition, in the existing MPEG-4, aP-picture is coded using a single, preceding reference picture appearingimmediately before it in display order, as reference, while in coding aB-picture, a single preceding reference picture appearing immediatelybefore, and a single subsequent reference picture appearing immediatelyafter, the current picture to be coded in display order, are both usedas reference pictures. For example, when the current picture to be codedis the picture B6, the two reference pictures coded immediately beforethe current picture to be coded are the picture P1 and the picture P4,with the reference picture coded later among the two being the pictureP4. Accordingly, when the current picture to be coded is the picture B6,the pixel values of the picture P1, and the picture P4, and the motionvectors obtained from the coding of the picture P4, are stored. Thepicture B6 shall be coded using the picture P1 and picture P4, storedwithin the memory, as reference pictures. The picture P1 is thereference picture immediately preceding the picture B6 in display order,and the picture P4 is the reference picture immediately following thepicture B6 in display order. Furthermore, during the coding in directmode of a current block to be coded within the picture B6, the motionvectors of a block, co-located with the current block to be coded,within the reference picture immediately following the picture B6 indisplay order (in other words, the picture P4), are stored within thememory, and shall be used to perform motion compensation. However, sincethe picture B6 is not referred to by other pictures, its pixel values,as well as motion vectors are not stored within the memory, and thememory is not updated.

When the picture P7 is the current picture to be coded, it is codedusing either the picture P1 or the picture P4, stored within the memory,as a reference picture. Since the picture P7 may be referred to by otherpictures, its pixel values and motion vectors shall be stored within thememory. In this instance, the pixel values and motion vectors storedwithin the memory are updated in accordance with the coding and decodingof the picture P7. Among the pixel values of the picture P1 and thepicture P4 stored within memory prior to the coding of the picture P7,the pixel values of the older picture, the picture P1, are removed, andthe new pixel values of the picture P7 are stored. In addition, themotion vectors of the picture P4 stored within the memory are removed,and the new motion vectors of the picture P7 are stored.

When the picture B8 is coded, two pictures, the picture P4 appearingbefore the picture B8, and the picture P7 appearing after the picture B8in display order, are referred to. When a current block within thepicture B8 is to be coded in direct mode, motion compensation isperformed using the motion vector of a block, co-located with thecurrent block, within the reference picture immediately following thepicture B8 in display order (the picture P7). The pixel values of thetwo pictures (the picture P4, as well as the picture P7) referred to bythe picture B8, and the motion vector of the picture P7 for use indirect mode, are stored within the memory. In addition, since thepicture B8 is not referred to by other pictures, the data within thememory is not updated.

In coding the picture B9, as in the coding of the picture B8, it iscoded using the two pictures stored within the memory as referencepictures. Likewise, when coding a current block within the picture B9 indirect mode, it is also coded using the motion vectors of the picture P7stored within the memory.

In this manner, by storing into the memory the pixel values of the tworeference pictures, and the motion vectors of only the reference pictureappearing subsequently among the two reference pictures mentioned above,even during the coding in direct mode of a current block within aB-picture, the coding can be performed without problems, in the existingMPEG-4. However, in the MPEG-4 format, since the reference pictures arefixed or few in number, the search area for finding a reference blockthat further improves coding efficiency for the current block islimited. As a result, coding efficiency is restricted, and satisfyingthe demands for low bit rate, high picture quality moving picturetransmission becomes difficult. For this reason, in the H.26L standard,the number of reference pictures, being the search area, is increased,and the reference pictures actually used as reference by each currentblock are set to be identified by reference indices. Then, as shown inFIG. 3, for each current block within a B-picture, the first referenceindex and second reference index, identifying the reference picturesused as reference, are set to be described within a bit stream. On theother hand, the assignment of reference indices for each referencepicture, as previously explained, is set by default, to assign the value“0” to the reference picture appearing immediately before and thereference picture appearing immediately after the current picture to becoded in display order, and to assign reference indices increasing invalue as the reference picture appears further off in display order,from the current picture to be coded. In this default setting, thereference index with a smaller value may be used to identify thereference picture that is closer in display order, to the picturecontaining the current block. However, the value of the reference indexused in identifying reference pictures increases as the referencepicture appears further away in display order, from the picturecontaining the current block. As a result, in the case where referencepictures appearing far away in display order, from the picturecontaining the current block are referred to by a large number ofcurrent blocks, within the default settings, reference indices withlarge values shall be described within the bit streams of a large numberof blocks, thereby increasing the data size of bit streams, which is notdesirable. Accordingly, remapping is accepted to allow the assignment ofsmaller reference index values to reference pictures that betterincrease coding efficiency for each picture.

In the present embodiment, a P-picture uses as reference pictures, fourI-pictures or P-pictures appearing ahead of the current picture to becoded in display order, and a B-picture uses as reference, fourI-pictures or P-pictures appearing ahead of the current picture to becoded in display order, and a single I-picture or P-picture appearingafter the current picture to be coded in display order. As shown in FIG.17B, for the coding of a B-picture in the present embodiment, the pixelvalues of the five reference pictures coded immediately before eachpicture, are stored within the memory, and the motion vector of only thereference picture coded immediately before the current picture to becoded is stored. For example, every time the motion vector estimationunit 108 estimates motion vectors for each current block in the currentpicture to be coded, the mode selection unit 109 temporarily stores theestimated motion vectors in an area different from the motion vectorstorage unit 116 where other motion vectors are already stored. In thecase where the subject current picture to be coded is a P-picture, themode selection unit 109 overwrites the previously stored motion vectorswith the motion vectors stored temporarily in a different storage area.In addition, at the start of coding for the next current block to becoded, the motion vectors of a new current picture to be coded areoverwritten into the storage area where the motion vectors of the mostrecently coded picture are temporarily stored, with the overwrittenmotion vectors being stored temporarily. However, in the case where thesubject current picture to be coded is not a P-picture, the modeselection unit 109 deletes the motion vectors temporarily stored in themotion vector storage unit 116 after coding of all blocks of a currentpicture to be coded, for example. The method for deleting the motionvectors temporarily stored within the motion vector storage unit 116 maybe a method that erases motion vectors stored temporarily from themotion vector storage unit 116, or a method that directly overwrites themotion vectors of the next current picture to be coded into the storagearea where motion vectors are temporarily stored. Through these methods,the mode selection unit 109 controls the storage size of motion vectorsstored in the motion vector storage unit 116. Furthermore, in the codingof a B-picture in direct mode, in the case where the motion vectors ofthe second reference index having a value as “0” is not stored in thememory, the method already explained in the first embodiment shall beused to perform motion compensation.

In FIG. 17B, for example, in the case where the current picture to becoded is the picture B6, the picture I0, the picture P1, and the pictureP4, are coded first before the picture B6. Accordingly, at the time thepicture B6 is coded, the pixel values of the picture I0, the picture P1,as well as the picture P4, are stored within the memory. In addition, ifthere are I-pictures or P-pictures coded prior to the picture I0, thepixel values of the two pictures coded immediately before the picture I0are stored within the memory. The picture B6 is coded, using asreference, two reference pictures from among the reference picturesstored within the memory. Moreover, the motion vectors of the referencepicture coded immediately before the picture B6, namely the picture P4,are stored within the memory. In the case where a current block withinthe picture B6 is coded in direct mode, if the reference picture withthe second reference index as “0” for the picture B6 is the picture P4,the mode selection unit 109 performs motion compensation on the currentblock using the motion vectors stored within the memory. Furthermore, ifthe reference picture with the second reference index as “0” is not thepicture P4, for example, the motion vectors for the current block isassumed as “0”, the reference picture with the first reference indexvalue as “0” for the picture B6, and the reference picture with thesecond reference index as “0” for the picture B6, shall be used asreference and bi-predictive motion compensation is performed.

When the current picture to be coded is the picture P7, the picture P7is coded using as reference pictures, the three reference pictures codedimmediately before, and stored within the memory (the picture I0, thepicture P1, and the picture P4). Since the picture P7 may be referred toby other pictures, following the coding and decoding of the picture P7,the pixel values of the oldest reference picture stored within thememory are removed, and the new pixel values of the picture P7 arestored. In addition, the motion vectors of the picture P4 stored withinthe memory are removed, and the new motion vectors for the picture P7are stored.

Next, when the picture B8 is coded, two out of the five referencepictures stored within the memory are used as reference pictures, andthe coding of the picture B8 is performed. In addition, in the casewhere a current block within the picture B8 is coded in direct mode, ifthe reference picture with the second reference index as “0” for thepicture B8 is the picture P7, the mode selection unit 109 uses themotion vectors of the picture P7 stored within the memory to performmotion compensation for the current block. If the reference picture witha second reference index as “0” for the picture B8 is not the pictureP7, the motion vectors of the current block is assumed as “0”, and tworeference pictures (the picture P4 and the picture P7), the referencepicture with the first reference index value as “0” and the referencepicture with the second reference index as “0” for the picture B6, shallbe used as reference, and bi-predictive motion compensation isperformed. Since the picture B8 is not referred to by other pictures, itis not stored in memory and the data within the memory is not updated.In the case where the picture B9 is coded, the coding shall be the sameas in the case of coding for the picture B8.

In this manner, according to the aforementioned third embodiment, bylimiting the motion vectors to be stored within the memory for thecoding of a B-picture in direct mode to the motion vectors of the singlereference picture coded immediately before the B-picture, it is possibleto provide a moving picture coding apparatus that enables theimprovement of coding efficiency while controlling the memory capacityrequired for the coding of moving pictures.

Furthermore, in the above-mentioned third embodiment, the moving picturecoding apparatus 900 generates video data, as well as control codes forrecording on the recording medium 400. However, this may not be limitedto generation for recording on a recording medium. For example, thegeneration of video streams distributed for digital televisionbroadcasts, and so on, and management information for the management ofthe reproduction of such video streams, is also possible. FIG. 18 is adiagram showing an example of a broadcasting packet sent out duringdigital television broadcasts. In the same diagram, parts represented byan “S”, “A”, and “V”, indicate management information, audio data, andvideo data packets, respectively. In digital television broadcasts,coded data such as video streams, as well as audio streams, arepartitioned into packets of appropriate data lengths (i.e., 188 bytesfor an MPEG-2 transport stream), and transmitted on a per-packet basis.Audio data packets, as well as video data packets are insertedthroughout with management information packets, and transmitted. The bitstreams generated by the moving picture coding apparatus 900 aretransmitted as video data packets, like those mentioned above. Moreover,the control codes generated by the coding control unit 910 are describedwithin the management information and transmitted as managementinformation packets. In this manner, by transmitting the bit streamsgenerated by the moving picture coding apparatus 900 as video datapackets, and transmitting the control codes generated by the codingcontrol unit 910 as management information packets, the reproduction ofmoving pictures of high picture quality and high coding efficiency whileeconomizing on memory usage becomes possible, even in a reproductionapparatus such as a set top box (STB) and a digital television thatreceives digital television broadcasts and reproduces video data.

Furthermore, in the above-mentioned third embodiment, control codes areexplained under the assumption that they are information indicating thatsome limitation has been set on the data to be stored within the memory.In the present invention, however, control codes are not limited tobeing such, and may also be information merely indicating that a bitstream generated, was coded under some form of limitation. In addition,the control codes may also be flags, or the like, indicating, forexample “no limitations” when the value is “0”, and “with limitations”when the value is “1”. Moreover, the control codes may also be codesindicating “with limitations”, in the presence of descriptions in aspecified area within management information, or “no limitations”, inthe absence of such descriptions. Of course, it goes without sayingthat, by referring to a table stored in the memory beforehand, thecontrol codes may also be ones that enable the identification ofmultiple types of processing, such as the presence of some limitationset on data to be stored in memory, or the number of P-pictures codedimmediately before the current picture to be coded, whose motion vectorsare to be stored within the memory, and so on.

Fourth Embodiment

Hereinafter, a reproduction apparatus for reproducing an AV streamrecorded on a recording medium, as well as a moving picture decodingapparatus included in the said recording apparatus shall be explained.FIG. 19 is a block diagram showing the outline configuration of areproduction apparatus 1500 in the fourth embodiment. The reproductionapparatus 1500 is a reproduction apparatus that reads out a coded AVstream, and management information for managing the reproduction of theAV stream, from the recording medium 400, and reproduces video dataaccompanied by sound. It includes a video decoding unit 1501, an audiodecoding unit 1502, a system decoding unit 1503, a monitor 1511, as wellas a speaker 1512. The system decoding unit 1503 reads out managementinformation and a coded AV stream from the recording medium, andextracts control codes from the management information for output to thevideo decoding unit 1501. In addition, the video stream and audio streamread out are separated and outputted to the video decoding unit 1501 andthe audio decoding unit 1502, respectively. In the case where it isindicated in the control codes obtained from the system decoding unit1503 that a limitation has been set for the data to be stored within thememory, during the decoding of the video stream obtained from the systemdecoding unit 1503, the video decoding unit 1501 stores into the memory,the motion vectors of the single reference picture decoded immediatelybefore the current picture to be decoded, and decodes the currentpicture to be decoded. The video decoding unit 1501 decodes each pictureincluded in the video stream, outputs each decoded picture in displayorder, and outputs a video signal for displaying a moving picture to themonitor 1511. The audio decoding unit 1502 decodes the audio streamobtained from the system decoding unit 1503, generates an audio signal,and outputs the generated audio signal to the speaker 1512. The monitor1511 displays the video signal generated by the video decoding unit1501. The speaker 1512 reproduces the audio signal generated by theaudio decoding unit 1502 in the form of sound.

FIG. 20 is a block diagram showing the configuration of a moving picturedecoding apparatus 1600 that corresponds to the video decoding unit 1501shown in FIG. 19. The moving picture decoding apparatus 1600 is a movingpicture decoding apparatus that, in the case where it is indicated inthe control codes inputted from an external source that a limitation hasbeen set for the storage of data within the memory, stores into thememory the motion vectors of the single reference picture decodedimmediately before the current picture to be decoded, and decodes a bitstream. It includes a prediction error decoding unit 702, a motioncompensation decoding unit 705, a motion vector storage unit 706, aframe memory 707, an addition unit 708, a switch 709, a switch 710, abit stream analysis unit 1601, as well as a mode decoding unit 1603. Inthe same diagram, since the structural components that are the same asthose in the moving picture decoding apparatus 700 shown in FIG. 11 arealready explained, the same reference numbers shall be assigned and theexplanations shall not be repeated. The bit stream analysis unit 1601outputs control codes inputted from an external source, to the modedecoding unit 1603. The bit stream analysis unit 1601 extracts modeselection information, motion vector information, and so on, from aninputted bit stream. It outputs the extracted mode selection informationto the mode decoding unit 1603, and the motion vector information to themotion compensation decoding unit 705. In addition, it outputsprediction error coding data to the prediction error decoding unit 702.The mode decoding unit 1603 controls the motion compensation decodingunit 705 to store into the motion vector storage unit 706, the motionvectors of the single reference picture decoded immediately before thecurrent picture to be decoded, in the case were it is indicated in thecontrol codes inputted from the bit stream analysis unit 1601, that alimitation has been set for the data to be stored within the memory, orin the case where it is indicated that the inputted video stream wascoded under some form of limitation.

For the moving picture decoding apparatus 1600, the number of motionvectors to be stored in the motion vector storage unit 706 and thenumber of pixel values to be stored into the frame memory 707, shall bethe same as the example shown in FIG. 17B.

As described above, with the moving picture decoding apparatus 1600 inthe fourth embodiment, even in the case where a video stream recorded ona recording medium is decoded, the effect of being able to decode a bitstream with a high coding efficiency is achieved, while economizing onthe memory resources of a moving picture decoding apparatus.

In addition, in the above-mentioned fourth embodiment, the case wherethe moving picture decoding apparatus 1600 decodes a video streamrecorded on a recording medium is explained. However, in the presentinvention, it is not limited as such. The moving picture decodingapparatus in the present invention may also decode a video stream withina broadcast packet distributed for digital television broadcasts. Inthis case, the moving picture decoding unit 1600 is included in adigital television (DTV) or STB, and so on, that receive digitaltelevision broadcasts, for example. The broadcast packet shown in FIG.18 is received by DTV, STB, and so on, and is separated into individualmanagement information, audio data, as well as video data packets. Fromthe separated management information, control codes indicating that somelimitation has been set on the data to be stored within the memory, isfurther extracted and inputted to the moving picture decoding apparatus1600. In addition, the video streams and audio streams extracted fromeach packet are inputted to the moving picture decoding apparatus 1600and audio decoding apparatus, and decoded synchronously, in real time.Since the configuration, as well as the operation of such moving picturedecoding apparatus 1600 is the same as that of the moving picturedecoding apparatus 1600 shown in FIG. 20, the explanation shall not berepeated.

As described above, with the moving picture decoding apparatus 1600 inthe present embodiment, even in the case where a video stream within abroadcast packet distributed for digital television broadcasts, and thelike, is coded, the effect of being able to decode a bit stream with ahigh coding efficiency is achieved, while economizing on the memoryresources of a moving picture decoding apparatus.

Furthermore, even in the above-mentioned fourth embodiment, controlcodes shall not be limited to the case where a limitation being set fordata to be stored within memory is indicated. They may also just beinformation merely indicating that a bit stream generated, was codedunder some form of limitation. Moreover, these control codes may also beflags, or the like, indicating, for example “no limitations” when thevalue is “0”, and “with limitations” when the value is “1”. Moreover,these control codes may also be codes indicating “with limitations”, inthe presence of descriptions in a specified area within the managementinformation, or “no limitations”, in the absence of such descriptions.Of course, it goes without saying that, by referring to a table storedin the memory beforehand, the control codes may also ones that enablethe identification of the presence of some limitation set on data to bestored in memory, or the number of P-pictures coded immediately beforethe current picture to be coded, whose motion vectors are to be storedwithin the memory, and so on.

Fifth Embodiment

Each picture, composing a moving picture, signifies a single unit forcoding that implies both a frame and a field. In the previouslymentioned first to fourth embodiments, the case where all pictures arein the frame structure is explained. However, even in the case wherepictures are handled on a per-field basis, the same effect can beattained by applying the invention explained above. Hereinafter, in thecase where the method explained in the aforementioned embodiment isapplied to interlaced pictures, an embodiment, different from thepreviously mentioned first to fourth embodiments, shall be considered.Here, an interlaced picture is a picture where one frame is constructedfrom two fields with different times (time stamps). In the coding anddecoding of an interlaced picture, it is possible to process a singleframe directly as a frame, or as two fields, or to process respectiveblocks within a frame, in the frame structure or field structure.

Furthermore, whether each picture shall be coded in the frame structureor field structure may, for example, be decided on adaptively by thecoding control unit 110, shown in FIG. 5. In determining whether to codein the frame structure or field structure, a method exists, for example,where the variance of pixel values within a picture are obtained by theframe structure and field structure, with the structure with the smallervariance being selected. Moreover, although a method in which eachpicture is coded either in the frame structure or field structure, on aper-block basis may also be considered, the case of switching betweenframe structure and field structure, on a per-picture basis, shall beexplained here. First, the assignment of picture numbers and referenceindices in the case where pictures in the field structure are includedin a moving picture shall be explained, after which, specificexplanation of the fifth embodiment shall be done.

In FIG. 2, picture numbers and reference indices are shown in the casewhere all pictures composing a moving picture are in the framestructure. However, using FIG. 21 and FIG. 22, the case where picturesmaking up a moving picture are of mixed frame and field structure shallbe explained.

FIG. 21 is a diagram showing an example of the reference relationshipbetween pictures in the case where the current picture to be coded ordecoded is in the field structure. Hereinafter, the field configuredfrom the odd numbered lines of a frame shall be known as a “top field”,and the field configured from the even numbered lines of a frame shallbe known as a “bottom field”. In addition, in the present embodiment,the case is explained where the top field is the field ahead, in thedisplay order of pictures in the field structure, and the bottom fieldis the field behind, in the display order of pictures in the fieldstructure. However, even if these sequences are switched, the sameeffect may be attained through the same operation as that in the presentinvention. In the case where the current picture to be coded or decodedis in the field structure, all the reference pictures shall be handledin the field structure. FIG. 21 shows the case where a field B32 is thecurrent picture to be coded or decoded in the case where a picture B3 iscoded or decoded in the field structure, and the case where a pictureP1, B3, and P4 are coded or decoded in the field structure and a pictureB2 is coded or decoded in the frame structure, wherein the referencepicture B2 in the frame structure, is treated as two fields, B21 andB22. In addition, picture numbers are assigned in the order of coding ordecoding. In the default setting, the first reference index is assignedto a picture in the following order of priority: the picture ahead ofthe current picture to be coded or decoded in display order, the pictureclose to the current picture to be coded or decoded in coding ordecoding order, the picture with the same parity as the current pictureto be coded or decoded. Using FIG. 21 to explain specifically, first,the value “0” is assigned as the reference index to the field B22, beinga field ahead of the field B32 in display order, having the same parity,and being the closest to the field B32 in coding or decoding order.Next, the value “1” is assigned as the reference index to the field B31,being a field ahead of the field B32 in display order, having differentparity but being the closest to the field B32 in coding or decodingorder.

Here, parity is a value indicating the first (top) field and second(bottom) field. In a bit stream of a moving picture, parity informationmay be coded or decoded, for example, on a per-slice basis

FIG. 22 is a diagram showing an example of the reference relationshipbetween pictures in the case where the current picture to be coded ordecoded is a frame. In the case where the current picture to be coded ordecoded is a frame, all reference pictures shall be treated as frames.As in FIG. 22, in the case of coding or decoding for the pictures B2 andP4 in the field structure, and the pictures P1 and B3 in the framestructure, the fields B21, B22, P41, P42 coded or decoded in the fieldstructure shall be treated as the two reference frames, B2 and P4.Moreover, the assignment of picture numbers and reference indices shallbe as explained in FIG. 2A and FIG. 2B.

FIG. 23 is a diagram showing the temporal sequence of respective framesduring the coding or decoding of a moving picture. In FIG. 23, theframes P1 and P4 are processed as P-pictures, and the frames B2 and B3are processed as B-pictures. In addition, a single frame may be treatedas 2 fields. For example, the frame P1 may be treated as the fields P11and P12, the frame B2 as the fields B21 and B22, the frame B3 as thefields B31 and B32, and the frame P4 as the fields P41 and P42.Furthermore, it is assumed that each frame is coded or decodedadaptively, in either the frame structure or field structure.

Hereinafter, the fifth embodiment shall be explained specifically. Thepoint of difference between the fifth embodiment and the previouslymentioned first to fourth embodiments is the storage of motion vectorsfor each parity, as a method for reducing the storage size of motionvectors that must be stored for direct mode.

In FIG. 23, coding or decoding is performed in the units assigned by thesymbols on the upper-row, among the symbols indicating pictures. Forexample, in FIG. 23, all pictures are processed on a per-field basis.First, direct mode in the case where pictures composing a moving pictureare treated in the field mode shall be explained using FIG. 23.

It is assumed that the current picture to be processed is the field B31.In other words, the frame B3 is processed in the field structure.Moreover, the field B31 shall use as reference pictures, the field P11,the field P12, the field P41, or the field P42. Coding or decoding forthese reference pictures is already completed. Furthermore, it isassumed that the frame P1 and the frame P4 are processed on a per-fieldbasis.

Now, the case where a block “a” in the field B31 is processed in directmode shall be considered. In this case, the motion vectors of the block“b”, being the block co-located with the block “a”, inside the fieldP41, being the reference picture with the second reference index as “0”(here, it is assumed that the reference index is assigned by default),is used. Hereinafter, these motion vectors shall be called “referencemotion vectors”.

Here, as in FIG. 23, the case where the block “b” is processed using themotion vector “A”, where this motion vector “A” refers to the field P11,shall be explained. In this case, using the motion vectors obtained by aspecified method of calculation from the reference motion vector “A”,motion compensation is performed for the block “a”, on the precedingreference field, the field P11 (the field pointed out by the referencemotion vector “A”), and the subsequent reference field, the field P41(the field where the block “b” belongs). In this case, the motionvectors to be used in the processing of the block “a” are assumed to bethe motion vector “B” for the field P11, and the motion vector “C” forthe field P41. In this instance, assuming the size of the motion vector“A” to be MV1, the size of the motion vector “B” to be MVf1, and thesize of the motion vector “C” to be MVb1, MVf1 and MVb1 can be obtainedthrough the Equation 3 and Equation 4, respectively.MVf1=N1×MV1/D1  (Equation 3)MVb1=−M1×MV1/D1  (Equation 4)

Hereinafter, the values of N1, M1, and D1 shall be known as scalingcoefficients. It is assumed that scaling coefficients are values set ona per-field basis. For example, in this case, the scaling coefficientscan be set based on the temporal distances between each field. Forexample, by setting the temporal distance from the field P11 to thefield P41 as D1, the temporal distance from the field P11 to the fieldB31 as N1, and the temporal distance from the field B31 to the field P41as M1, MVf1 and MVb1 shall be motion vectors parallel to MV. Here, as amethod of determining the values of scaling coefficients, a methodexists that generates values on both the coding-side and decoding sideaccording to the same rule (i.e., generating from time information (timestamp)). Another method exists where during coding, the value is set anddescribed as related information within a bit stream or as the attachedinformation of a bit stream, and then, during decoding, the scalingcoefficients are obtained from the bit stream or the attachedinformation of the bit stream. Then, Equations 3 and 4 may be used tocalculate MVf1 and MVb1, in the decoding of the blocks coded in directmode.

Now, the operation of the motion vector storage unit 116 shall beexplained. The number of motion vectors to be stored in the motionvector storage unit 116 is designated by the coding control unit 110.Moreover, here, the motion vectors of one top field and the motionvectors of one bottom field are be stored at a time.

In this case, for determining the motion vectors to be stored, there isa method that stores the motion vectors of the reference picture codedor decoded immediately before the current picture in the coding ordecoding order, among reference pictures. Here, the motion vectorsalready stored in the motion vector storage unit 116 may be overwritten,in sequence, each time a coding mode is determined.

Specifically, as shown in FIG. 24A, the motion vector storage unit 116is controlled in the same manner as a first-in, first-out (FIFO) memory,where the newest pictures shall be stored at all times. Memory contentshaving the oldest storage times are cancelled in sequence, as new itemsare stored. In addition, in FIG. 24A, the case where only the motionvectors used in the coding or decoding of an I-picture or P-picture arestored, is shown. This is because the case where only I-pictures orP-pictures being referred to by B-pictures processed using direct modeis presupposed. In cases like where a B-picture refers to anotherB-picture, the motion vectors used during the coding or decoding of theB-picture used as a reference picture, may also be stored.

In coding or decoding for the field B31, the motion vectors during thecoding or decoding of the field P41 are stored in the memory address M1,and it is possible to derive the motion vector “B”, and the motionvector “C” using the motion vector “A” stored in the memory address M1.In addition, the motion vectors during the coding or decoding of thefield P42 are stored in the memory address M2.

As another example for the motion vector storage unit 116, as shown inFIG. 24B, the motion vectors of two top fields and the motion vectors oftwo bottom fields may also be stored at a time. In this case, the motionvector storage unit 116 is controlled in the same manner as a first-in,first-out (FIFO) memory, and the motion vectors stored in the memoryaddress M11 is transferred to the memory address M12 if motion vectorsshall be stored anew. The motion vectors stored in the memory addressM12 shall be deleted. It is the same for the memory addresses M21 andM22.

Moreover, in the case where storage areas storing motion vectors for aplurality of pictures on a per-parity basis exist, especially withregard to areas for unnecessary pictures, it is also possible tospecifically release such areas for use. In addition, the physicallayout of the memory to be used need not be on a sequential layout. Theusage of memory area may be determined based on a FIFO-type operation.

Furthermore, in the explanation for FIG. 24A and FIG. 24B, the casewhere the current pictures to be coded or decoded, are all fields isdescribed. However, in the case where the current picture to be coded ordecoded is a frame, the motion vectors used during motion compensationfor such current picture to be coded or decoded may be stored in thememory address M1 and memory address M2, in FIG. 24A. Likewise, in FIG.24B, they may be stored in the memory addresses M11 and M21. In the casewhere the current picture to be coded or decoded is a field and thereference picture is a frame, the reference picture shall be treated asa field. Furthermore, in direct mode, the motion vectors of thereference field with the reference index as “0” is used. However in thiscase, if the reference field with the reference index as “0” wasoriginally processed in the frame structure, the two fields included insuch frame shall be considered as having the same motion vectors asthose in the frame structure. As such, by storing the motion vectorsprocessed in the frame structure in both the memory addresses M1 and M2,it is possible to obtain the motion vectors needed for use in directmode without concern as to whether the reference picture was processedin the field structure or frame structure.

On the other hand, in the case where a picture, the same as the framebeing the reference picture, is not treated as being in either the topfield or bottom field, in other words, for example, in the case wherethe motion vectors used during the motion compensation of the framebeing the reference picture, when such frame is the current frame to becoded or decoded, are stored in the memory address for the top field,there is a possibility of mistaking the motion vector to be referred todue to the reassignment of reference indices (since changing referenceindices (remapping) is possible). Moreover, in this way, even in thecase where a picture, the same as the frame being the reference picture,is not treated as being in either the top field or bottom field, if thememory address for storing motion vectors is for one picture, as shownin FIG. 24A, the problem of mistaking the motion vector to be referredto should not arise often even if reference indices are re-assigned.

In this manner, by controlling so that motion vectors are stored on afield parity basis, the motion vectors necessary for direct mode shallnot be overwritten, even in the case where a picture is processed in thefield structure.

In addition, it is possible to adaptively combine the present inventionexplained in the aforementioned first and second embodiments with thecontrolling for the storage of motion vectors on a field parity basis.Furthermore, by controlling in such a way that motion vectors are storedon a field parity basis, the number of motion vectors to be stored islimited, and a reduction in memory size is made possible. Also, even inthe case where a picture is processed in the field structure, there isno overwriting of the motion vectors necessary for direct mode. And, themotion vectors used during the coding or decoding of the two fields (topfield and bottom field) belonging to the same frame, may be stored asone pair. Whether fields belong to the same frame or not may bedetermined from information showing, for example, the continuity ofpicture numbers and parity. Alternatively, it can be determined frominformation showing the continuity of the frame P1, the frame B2, theframe B3, and the frame P4. Furthermore, although the case whereswitching from coding or decoding in either the frame structure or fieldstructure, done on a per-picture basis, is explained in the presentembodiment, switching from coding or decoding in either the framestructure or field structure may also be done on a per-block basiswithin a picture. In this case, the motion vectors of a block processedin the frame structure are stored in both the top field motion vectormemory and the bottom field motion vector memory, and the motion vectorsfor blocks in the field structure may be stored in the motion vectormemory for the applicable parity.

Sixth Embodiment

In addition, by recording a program for realizing the layout of themoving picture coding method or the moving picture decoding method asshown in each of the above-mentioned embodiments, on a storage mediumsuch as a flexible disk, it becomes possible to perform the processingas shown in each of the above embodiments easily in an independentcomputer system.

FIG. 25A to 25C are explanatory illustrations of a storage medium forthe storage of a program for realizing the moving picture coding methodand the moving picture decoding method in the first embodiment to fifthembodiments by a computer system.

FIG. 25B shows the front view of a flexible disk and the schematiccross-section, as well as a flexible disk, and FIG. 25A shows an exampleof a physical format of the flexible disk as a recording medium itself.The flexible disk FD is contained in a case F, a plurality of tracks Trare formed concentrically on the surface of the said disk in the radiusdirection from the periphery, and each track is divided into 16 sectorsSe in the angular direction. Therefore, in the flexible disk storing theabove-mentioned program, the moving picture coding method and the movingpicture decoding method as the above program are recorded in an areaallocated for it on the above flexible disk FD.

In addition, FIG. 25C shows the configuration for recording andreproducing the program on and from the flexible disk FD. For recordingthe program on the flexible disk FD, the computer system Cs writes themoving picture coding method and the moving picture decoding method asthe program on the flexible disk FD via a flexible disk drive FDD. Forconstructing the above moving picture coding method and the movingpicture decoding method in the computer system by the program recordedon the flexible disk, the program is read out from the flexible disk viathe flexible disk drive and transferred to the computer system.

Note that the above explanation is made on the assumption that arecording medium is a flexible disk, but the same processing can also beperformed using an optical disk. In addition, the recording medium isnot limited to these, but any other mediums such as a CD-ROM, memorycard, and a ROM cassette can be used in the same manner if a program canbe recorded on them.

Seventh Embodiment

Further, the applications of the moving picture coding method and themoving picture decoding method as shown in the above embodiments and asystem using them shall be explained here.

FIG. 26 is a diagram showing the overall configuration of a contentproviding system ex100 for realizing content distribution service. Thearea for providing communication service is divided into cells ofdesired size, and base stations ex107 to ex110 which are fixed wirelessstations are placed in respective cells.

In this content providing system ex100, various devices such as acomputer ex111, a personal digital assistant (PDA) ex112, a cameraex113, a mobile phone ex114 and a camera-equipped mobile phone ex115 areconnected to the Internet ex101, via an Internet service provider ex102,a telephone network ex104 and base stations ex107 to ex110, for example.

However, the content providing system ex100 is not limited to thecombination as shown in FIG. 26, and may be connected to a combinationof any of them. Also, each device may be connected directly to thetelephone network ex104, not through the base stations ex107 to ex110which are the fixed wireless stations.

The camera ex113 is a device such as a digital video camera capable ofshooting moving pictures. The mobile phone may be any of a mobile phoneof a Personal Digital Communications (PDC) system, a Code DivisionMultiple Access (CDMA) system, a Wideband-Code Division Multiple Access(W-CDMA) system or a Global System for Mobile Communications (GSM)system, a Personal Handyphone System (PHS) and the like.

Also, a streaming server ex103 is connected to the camera ex113 via thebase station ex109 and the telephone network ex104, which enables livedistribution or the like using the camera ex113 based on the coded datatransmitted from the user. The coding of the data shot by the camera maybe performed by the camera ex113, the server for transmitting the data,or the like. Also, the moving picture data shot by a camera ex116 may betransmitted to the streaming server ex103 via the computer ex111. Thecamera ex116 is a device such as a digital camera capable of shootingstill and moving pictures. In this case, either the camera ex116 or thecomputer ex111 may code the moving picture data. An LSI ex117 includedin the computer ex111 or the camera ex116 performs coding processing.Note that software for coding and decoding moving pictures may beintegrated into any type of a storage medium (such as a CD-ROM, aflexible disk and a hard disk) that is a recording medium which can beread by the computer ex111 or the like. Furthermore, the camera-equippedmobile phone ex115 may transmit the moving picture data. This movingpicture data is the data coded by the LSI included in the mobile phoneex115.

In this content providing system ex100, contents (such as a video of alive music performance) shot by users using the camera ex113, the cameraex116 or the like are coded in the same manner as in the aboveembodiments and transmitted to the streaming server ex103, while thestreaming server ex103 makes stream distribution of the above contentdata to the clients at their request. The clients include the computerex111, the PDA ex112, the camera ex113, the mobile phone ex114, and soon, capable of decoding the above-mentioned coded data. The contentproviding system ex100 is a system in which the clients can thus receiveand reproduce the coded data, and further can receive, decode andreproduce the data in real time so as to realize personal broadcasting.

When each device in this system performs coding or decoding, the movingpicture coding apparatus or the moving picture decoding apparatus asshown in each of the above-mentioned embodiments may be used.

A mobile phone will be explained as an example thereof.

FIG. 27 is a diagram showing a mobile phone ex115 which uses the movingpicture coding method and the moving picture decoding method asexplained in the above embodiments. The mobile phone ex115 has anantenna ex201 for sending and receiving radio waves between the basestation ex110, a camera unit ex203 such as a CCD camera capable ofshooting video and still pictures, a display unit ex202 such as a liquidcrystal display for displaying the data obtained by decoding video shotby the camera unit ex203, video received by the antenna ex201, or thelike, a main body including a set of operation keys ex204, a voiceoutput unit ex208 such as a speaker for outputting voices, a voice inputunit ex205 such as a microphone for inputting voices, a storage mediumex207 for storing coded or decoded data, such as data of moving or stillpictures shot by the camera, and data of text, moving pictures or stillpictures of received e-mails, and a slot unit ex206 for attaching thestorage medium ex207 into the mobile phone ex115. The storage mediumex207 includes a flash memory element, a kind of Electrically Erasableand Programmable Read Only Memory (EEPROM) that is an electricallyrewritable and erasable nonvolatile memory, in a plastic case such as anSD card.

Further, the mobile phone ex115 will be explained with reference to FIG.28. In the mobile phone ex115, a power supply circuit unit ex310, anoperation input control unit ex304, a picture coding unit ex312, acamera interface unit ex303, an Liquid Crystal Display (LCD) controlunit ex302, a picture decoding unit ex309, a multiplex/demultiplex unitex308, a record/reproduce unit ex307, a modem circuit unit ex306 and avoice processing unit ex305, are connected to a main control unit ex311,and to each other, via a synchronous bus ex313. The main control unitex311 is for the overall controlling of each unit of the main bodyincluding the display unit ex202 and the operation keys ex204.

When a call-end key or a power key is turned ON by a user's operation,the power supply circuit unit ex310 supplies respective units with powerfrom a battery pack so as to activate the camera-equipped digital mobilephone ex115 to a ready state.

In the mobile phone ex115, under the control of the main control unitex311 including a CPU, ROM, RAM and the like, the voice processing unitex305 converts the voice signals received by the voice input unit ex205in conversation mode into digital voice data, the modem circuit unitex306 performs spread spectrum processing of the digital voice data, andthe send/receive circuit unit ex301 performs digital-to-analogconversion and frequency transformation of the data, so as to transmitthe result via the antenna ex201. Also, in the mobile phone ex115, thedata received by the antenna ex201 in conversation mode is amplified andperformed of frequency transformation and analog-to-digital conversion,the modem circuit unit ex306 performs inverse spread spectrum processingof the data, and the voice processing unit ex305 converts it into analogvoice data, so as to output the result via the voice output unit ex208.

Furthermore, when transmitting an e-mail in data communication mode, thetext data of the e-mail inputted by operating the operation keys ex204on the main body is sent out to the main control unit ex311 via theoperation input control unit ex304. In the main control unit ex311,after the modem circuit unit ex306 performs spread spectrum processingof the text data and the send/receive circuit unit ex301 performsdigital-to-analog conversion and frequency transformation of it, theresult is transmitted to the base station ex110 via the antenna ex201.

When picture data is transmitted in data communication mode, the picturedata shot by the camera unit ex203 is provided to the picture codingunit ex312 via the camera interface unit ex303. When the picture data isnot transmitted, the picture data shot by the camera unit ex203 can alsobe displayed directly on the display unit 202 via the camera interfaceunit ex303 and the LCD control unit ex302.

The picture coding unit ex312, including the picture coding apparatusexplained in the present invention, compresses and codes the picturedata provided from the camera unit ex203 by the coding method used forthe picture coding apparatus as shown in the above-mentioned embodimentsso as to transform it into coded picture data, and sends it out to themultiplex/demultiplex unit ex308. At this time, the mobile phone ex115sends out the voices received by the voice input unit ex205 duringpicture pickup by the camera unit ex203, to the multiplex/demultiplexunit ex308 as digital voice data via the voice processing unit ex305.

The multiplex/demultiplex unit ex308 multiplexes the coded picture dataprovided from the picture coding unit ex312 and the voice data providedfrom the voice processing unit ex305 by a predetermined method, and themodem circuit unit ex306 then performs spread spectrum processing of theresulting multiplexed data, and the send/receive circuit unit ex301performs digital-to-analog conversion and frequency transformation onthe result, for transmitting via the antenna ex201.

As for receiving data of a moving picture file which is linked to aWebsite or the like in data communication mode, the modem circuit unitex306 performs inverse spread spectrum processing of the data receivedfrom the base station ex110 via the antenna ex201, and sends out theresulting multiplexed data to the multiplex/demultiplex unit ex308.

In order to decode the multiplexed data received via the antenna ex201,the multiplex/demultiplex unit ex308 demultiplexes the multiplexed datainto a coded bit stream of picture data and a coded bit stream of voicedata, and provides the coded picture data to the picture decoding unitex309 and the voice data to the voice processing unit ex305respectively, via the synchronous bus ex313.

Next, the picture decoding unit ex309, including the picture decodingapparatus explained in the present invention, decodes the coded bitstream of the picture data by the decoding method paired with the codingmethod as shown in the above-mentioned embodiments, so as to generatereproduced moving picture data, and provides this data to the displayunit ex202 via the LCD control unit ex302, and thus moving picture dataincluded in a moving picture file linked to a Website, for instance, isdisplayed. At the same time, the voice processing unit ex305 convertsthe voice data into analog voice data, and provides this data to thevoice output unit ex208, and thus voice data included in a movingpicture file linked to a Website, for instance, is reproduced.

The present invention is not limited to the above-mentioned system.Satellite or ground-based digital broadcasting has been in the newslately, and at least either the picture coding apparatus or the picturedecoding apparatus in the above-mentioned embodiments can beincorporated into such a digital broadcasting system as shown in FIG.29. More specifically, a coded bit stream of video information istransmitted from a broadcast station ex409 to a communication orbroadcast satellite ex410 via radio waves. Upon receipt of it, thebroadcast satellite ex410 transmits radio waves for broadcasting, a homeantenna ex406 with a satellite broadcast reception function receives theradio waves, and an apparatus such as a television (receiver) ex401 or aset top box ex407 decodes the coded bit stream for reproduction. Thepicture decoding apparatus as shown in the above-mentioned embodimentscan be implemented in a reproduction device ex403 for reading anddecoding a coded bit stream recorded on a storage medium ex402 such as aCD and DVD that is a recording medium. In this case, the reproducedvideo signals are displayed on a monitor ex404. It is also conceived toimplement the moving picture decoding apparatus in the set top box ex407connected to a cable ex405 for cable television or the antenna ex406 forsatellite and/or ground-based broadcasting so as to reproduce them on amonitor ex408 of the television. The moving picture decoding apparatusmay be incorporated into the television, not in the set top box. Or, acar ex412 having an antenna ex411 which can receive signals from thesatellite ex410, the base station ex107 or the like, for reproducingmoving pictures on a display apparatus such as a car navigation deviceex413 or the like in the car ex412.

Furthermore, the picture coding apparatus as shown in theabove-mentioned embodiments can code picture signals for recording themon a recording medium. As a concrete example, there is a recorder ex420such as a DVD recorder for recording picture signals on a DVD disk ex421and a disk recorder for recording them on a hard disk. They can also berecorded on an SD card ex422. If the recorder ex420 includes the movingpicture decoding apparatus as shown in the above-mentioned embodiments,the picture signals recorded on the DVD disk ex421 or the SD card ex422can be reproduced for display on the monitor ex408.

Note that as the configuration of the car navigation device ex413, theconfiguration without the camera unit ex203, the camera interface unitex303 and the picture coding unit ex312, out of the units as shown inFIG. 28, is conceivable. The same applies to the computer ex111, thetelevision (receiver) ex401 and others.

Moreover, three types of implementations can be conceived for a terminalsuch as the above-mentioned mobile phone ex114; a sending/receivingterminal equipped with both an encoder and a decoder, a sending terminalequipped with an encoder only, and a receiving terminal equipped with adecoder only.

As described above, it is possible to use the moving picture codingmethod or the moving picture decoding method as shown in the aboveembodiments in any of above-mentioned devices and systems, and thus theeffects explained in the above embodiments can be obtained.

From the invention thus described, it will be obvious that theembodiments of the invention may be varied in many ways. Such variationsare not to be regarded as a departure from the spirit and scope of theinvention, and all such modifications as would be obvious to one skilledin the art are intended for inclusion within the scope of the followingclaims.

INDUSTRIAL APPLICABILITY

The moving picture coding apparatus according to the present inventionis useful as a moving picture coding apparatus included in a personalcomputer, a PDA, a broadcast station for digital broadcasts, a mobilephone, and the like, with a communication function.

In addition, the moving picture decoding apparatus according to thepresent invention is useful as a moving picture decoding apparatusincluded in a personal computer, a PDA, an STB that receives digitalbroadcasts, a mobile phone, and the like, with a communication function.

1. A moving picture coding method executed by a moving picture codingapparatus, for coding a moving picture with motion compensation using amotion vector, the moving picture coding apparatus including a storagearea for storing at least one motion vector used in motion compensationof a current picture to be coded, the moving picture being a sequence ofpictures, each of the pictures being made up of a plurality of blocks,the method comprising: a step of generating limitation informationindicating a limitation on an amount of motion vector data to be storedin the storage area in the case where the at least one motion vectorused in the motion compensation of the current picture to be coded isstored, in the storage area, for use when a picture to be coded afterthe current picture to be coded is coded in direct mode; a step ofcoding the generated limitation information; a step of judging whetherthe current picture to be coded is a top field or a bottom field, in thecase where the current picture to be coded is one of the fields of aninterlaced picture, the fields being composed of the top field and thebottom field; and a step of (i) storing, into a top field storage area,the at least one motion vector used in motion compensation of thecurrent picture to be coded, the top field storage area having a storageamount that is limited according to the limitation information, in thecase where the current picture to be coded is judged to be the topfield, and (ii) storing, into a bottom field storage area, the at leastone motion vector used in motion compensation of the current picture tobe coded, the bottom field storage area having a storage amount that islimited according to the limitation information, in the case where thecurrent picture to be coded is judged to be the bottom field, whereinthe top field storage area and the bottom field storage area areincluded in the storage area.
 2. The moving picture coding methodaccording to claim 1, wherein the motion vector to be stored into thetop field storage area and the motion vector to be stored into thebottom field storage area are the motion vectors used in coding of thetop field and the bottom field belonging to one frame.
 3. The movingpicture coding method according to claim 1, further comprising a step ofstoring the motion vectors used in motion compensation of the currentpicture to be coded, into the top field storage area and the bottomfield storage area, in the case where the current picture to be coded iscoded in a frame structure.
 4. The moving picture coding methodaccording to claim 1, wherein in the step of coding the limitationinformation, the limitation information is coded as header informationwithin a bit stream of the moving picture.
 5. The moving picture codingmethod according to claim 4, wherein the header information is one ofthe following: header information contained in a header of the entirebit stream; header information contained in a header of each picture;and header information contained in a header of each slice.
 6. Themoving picture coding method according to claim 1, wherein in the stepof coding the limitation information, the limitation information iscoded as management information which is different information from abit stream of a moving picture.
 7. The moving picture coding methodaccording to claim 1, further comprising a step of storing the at leastone motion vector into a storage area based on the limitationinformation.
 8. The moving picture coding method according to claim 1,wherein the limitation information is information indicating the numberof pictures or the number of macroblocks.
 9. The moving picture codingmethod according to claim 1, wherein the limitation information isinformation indicating the number of preceding pictures whose one ormore motion vectors are to be stored into a storage area, the precedingpictures preceding a picture containing the current block to be coded,either in picture coding order or picture display order.
 10. The movingpicture coding method according to claim 1, further comprising a step ofcoding field information indicating whether the current picture to becoded is the top field or the bottom field.
 11. A moving picturedecoding method executed by a moving picture decoding apparatus, fordecoding a moving picture with motion compensation using a motionvector, the moving picture decoding apparatus including a storage areafor storing a motion vector used in motion compensation of a currentpicture to be decoded, the moving picture being a sequence of pictures,each of the pictures being made up of a plurality of blocks, the methodcomprising: a step of extracting, from a bit stream, limitationinformation indicating a limitation on an amount of motion vector datato be stored in the storage area in the case where the at least onemotion vector used in the motion compensation of the current picture tobe decoded is stored, in the storage area, for use when a picture to bedecoded after the current picture to be decoded is decoded in directmode; a step of judging whether the current picture to be decoded is atop field or a bottom field, in the case where the current picture to bedecoded is one of the fields of an interlaced picture, the fields beingcomposed of the top field and the bottom field; and a step of (i)storing, into a top field storage area, the at least one motion vectorused in motion compensation of the current picture to be decoded, thetop field storage area having a storage amount that is limited accordingto the limitation information, in the case where the current picture tobe decoded is judged to be the top field, and (ii) storing, into abottom field storage area, the at least one motion vector used in motioncompensation of the current picture to be decoded, the bottom fieldstorage area having a storage amount that is limited according to thelimitation information, in the case where the current picture to bedecoded is judged to be the bottom field, wherein the top field storagearea and the bottom field storage area are included in the storage area.12. The moving picture decoding method according to claim 11, whereinthe motion vector to be stored into the top field storage area and themotion vector to be stored into the bottom field storage area are themotion vectors used in decoding of the top field and the bottom fieldbelonging to one frame.
 13. The moving picture decoding method accordingto claim 11, further comprising a step of storing the motion vectorsused in motion compensation of the current picture to be decoded, intothe top field storage area and the bottom field storage area, in thecase where the current picture to be decoded is coded in a framestructure.
 14. The moving picture decoding method according to claim 11,wherein the step of extracting further comprises extracting, from thebit stream, field information indicating whether the current picture tobe decoded is the top field or the bottom field , and wherein in thestep of storing the at least one motion vector, the at least one motionvector is stored into a storage area, based on the limitationinformation and the field information.
 15. The moving picture decodingmethod according to claim 14, wherein the limitation information isinformation indicating the number of pictures or the number ofmacroblocks.
 16. The moving picture decoding method according to claim14, wherein the limitation information is information indicating thenumber of preceding pictures whose one or more motion vectors are to bestored into a storage area, the preceding pictures preceding a picturecontaining the current block to be decoded, either in picture decodingorder or picture display order.
 17. The moving picture decoding methodaccording to claim 14, wherein the limitation information is included inheader information of the bit stream, the header information being oneof the following: header information contained in a header of the entirebit stream; header information contained in a header of each picture;and header information contained in a header of each slice.
 18. A movingpicture coding method for coding a moving picture with motioncompensation using a motion vector, the moving picture being a sequenceof pictures, each of the pictures being made up of a plurality ofblocks, the method comprising: a step of generating limitationinformation indicating a limitation on a storage amount of at least onemotion vector used in motion compensation of a current picture to becoded; a step of coding the generated limitation information; a step ofjudging whether the current picture to be coded is a top field or abottom field, in the case where the current picture to be coded is oneof the fields of an interlaced picture, the fields being composed of thetop field and the bottom field; and a step of storing, into a top fieldstorage area, the at least one motion vector used in motion compensationof the current picture to be coded and whose storage amount is limitedaccording to the limitation information, in the case where the currentpicture to be coded is judged to be the top field, and storing, into abottom field storage area, the at least one motion vector used in motioncompensation of the current picture to be coded and whose storage amountis limited according to the limitation information, in the case wherethe current picture to be coded is judged to be the bottom field, themotion vector being stored for use when a picture to be coded after thecurrent picture to be coded is coded in direct mode, wherein in the stepof storing the at least one motion vector, motion vectors of a number ofpictures which is less than the number of pictures that can be referredto by the current picture to be coded are stored into the top fieldstorage and the bottom field storage area, based on the limitationinformation.
 19. A moving picture decoding method for decoding a movingpicture with motion compensation using a motion vector, the movingpicture being a sequence of pictures, each of the pictures being made upof a plurality of blocks, the method comprising: a step of extracting,from a bit stream, limitation information indicating a limitation on astorage amount of at least one motion vector used in motion compensationof a current picture to be decoded; a step of judging whether thecurrent picture to be decoded is a top field or a bottom field, in thecase where the current picture to be decoded is one of the fields of aninterlaced picture, the fields being composed of the top field and thebottom field; and a step of storing, into a top field storage area, theat least one motion vector used in motion compensation of the currentpicture to be decoded and whose storage amount is limited according tothe limitation information, in the case where the current picture to bedecoded is judged to be the top field, and storing, into a bottom fieldstorage area, the at least one motion vector used in motion compensationof the current picture to be decoded and whose storage amount is limitedaccording to the limitation information, in the case where the currentpicture to be decoded is judged to be the bottom field, the motionvector being stored for use when a picture to be decoded after thecurrent picture to be decoded is decoded in direct mode, wherein in thestep of storing the at least one motion vector, motion vectors of anumber of pictures which is less than the number of pictures that can bereferred to by the current picture to be decoded are stored into the topfield storage and the bottom field storage area, based on the limitationinformation.